Session 07¶

Security And Deployment¶

By the end of this session we’ll have deployed our learning journal to a public server.

So we will need to add a bit of security to it.

We’ll get started on that in a moment

But First¶

Questions About the Homework?

class EntryEditForm(EntryCreateForm):
    id = HiddenField()

View the form online

@view_config(route_name='action', match_param='action=edit',
             renderer='templates/edit.jinja2')
def update(request):
    id = int(request.params.get('id', -1))
    entry = Entry.by_id(id)
    if not entry:
        return HTTPNotFound()
    form = EntryEditForm(request.POST, entry)
    if request.method == 'POST' and form.validate():
        form.populate_obj(entry)
        return HTTPFound(location=request.route_url('detail', id=entry.id))
    return {'form': form, 'action': request.matchdict.get('action')}

See this view online

{% extends "layout.jinja2" %}
{% block body %}
<article>
  <!-- ... -->
</article>
<p>
  <a href="{{ request.route_url('home') }}">Go Back</a> ::
  <a href="{{ request.route_url('action', action='edit', _query=(('id',entry.id),)) }}">
    Edit Entry</a>
</p>
{% endblock %}

View this template online

class User(Base):
    __tablename__ = 'users'
    id = Column(Integer, primary_key=True, autoincrement=True)
    name = Column(Unicode(255), unique=True, nullable=False)
    password = Column(Unicode(255), nullable=False)

    @classmethod
    def by_name(cls, name):
        return DBSession.query(cls).filter(cls.name == name).first()

View this model online

Securing An Application¶

We’ve got a solid start on our learning journal.

We can:

view a list of entries
view a single entry
create a new entry
edit existing entries

But so can everyone who visits the journal.

It’s a recipe for TOTAL CHAOS

Let’s lock it down a bit.

AuthN and AuthZ¶

There are two aspects to the process of access control online.

Authentication: Verification of the identity of a principal
Authorization: Enumeration of the rights of that principal in a context.

Think of them as Who Am I and What Can I Do

All systems with access control involve both of these aspects.

But many systems wire them together as one.

In Pyramid these two aspects are handled by separate configuration settings:

config.set_authentication_policy(AuthnPolicy())
config.set_authorization_policy(AuthzPolicy())

If you set one, you must set the other.

Pyramid comes with a few policy classes included.

You can also roll your own, so long as they fulfill the requried interface.

You can learn about the interfaces for authentication and authorization in the Pyramid documentation

We’ll be using two built-in policies today:

AuthTktAuthenticationPolicy: sets an expirable authentication ticket cookie.
ACLAuthorizationPolicy: uses an Access Control List to grant permissions to principals

Our access control system will have the following properties:

Everyone can view entries, and the list of all entries
Users who log in may edit entries or create new ones

By default, Pyramid uses no security. We enable it through configuration.

Open learning_journal/__init__.py and update it as follows:

# add these imports
from pyramid.authentication import AuthTktAuthenticationPolicy
from pyramid.authorization import ACLAuthorizationPolicy
# and add this configuration:
def main(global_config, **settings):
    # ...
    # update building the configurator to pass in our policies
    config = Configurator(
        settings=settings,
        authentication_policy=AuthTktAuthenticationPolicy('somesecret'),
        authorization_policy=ACLAuthorizationPolicy(),
        default_permission='view'
    )
    # ...

We’ve now informed our application that we want to use security.

By default we require the ‘view’ permission to see anything.

But we have yet to assign any permissions to anyone at all.

Let’s verify now that we are unable to see anything in the website.

Start your application, and try to view any page (You should get a 403 Forbidden error response):

(ljenv)$ pserve development.ini
Starting server in PID 84467.
serving on http://0.0.0.0:6543

Implementing Authz¶

Next we have to grant some permissions to principals.

Pyramid authorization relies on a concept it calls “context”.

A principal can be granted rights in a particular context

Context can be made as specific as a single persistent object

Or it can be generalized to a route or view

To have a context, we need a Python object called a factory that must have an __acl__ special attribute.

The framework will use this object to determine what permissions a principal has

Let’s create one

In the same folder where you have models.py and views.py, add a new file security.py

from pyramid.security import Allow, Everyone, Authenticated

class EntryFactory(object):
    __acl__ = [
        (Allow, Everyone, 'view'),
        (Allow, Authenticated, 'create'),
        (Allow, Authenticated, 'edit'),
    ]
    def __init__(self, request):
        pass

The __acl__ attribute of this object contains a list of ACEs

An ACE combines an action (Allow, Deny), a principal and a permission

Now that we have a factory that will provide context for permissions to work, we can tell our configuration to use it.

Open learning_journal/__init__.py and update the route configuration for our routes:

# add an import at the top:
from .security import EntryFactory
# update the route configurations:
def main(global_config, **settings):
    """ This function returns a Pyramid WSGI application.
    """
    # ... Add the factory keyword argument to our route configurations:
    config.add_route('home', '/', factory=EntryFactory)
    config.add_route('detail', '/journal/{id:\d+}', factory=EntryFactory)
    config.add_route('action', '/journal/{action}', factory=EntryFactory)

We’ve now told our application we want a principal to have the view permission by default.

And we’ve provided a factory to supply context and an ACL for each route.

Check our ACL. Who can view the home page? The detail page? The action pages?

Pyramid allows us to set a default_permission for all views.

But view configuration allows us to require a different permission for a view.

Let’s make our action views require appropriate permissions next

Open learning_journal/views.py, and edit the @view_config for create and update:

@view_config(route_name='action', match_param='action=create',
             renderer='templates/edit.jinja2',
             permission='create') # <-- ADD THIS
def create(request):
    # ...

@view_config(route_name='action', match_param='action=edit',
             renderer='templates/edit.jinja2',
             permission='edit') # <-- ADD THIS
def update(request):
    # ...

At this point, our “action” views should require permissions other than the default view.

Start your application and verify that it is true:

(ljenv)$ pserve development.ini
Starting server in PID 84467.
serving on http://0.0.0.0:6543

You should get a 403 Forbidden for the action pages only.

Implement AuthN¶

Now that we have authorization implemented, we need to add authentication.

By providing the system with an authenticated user, our ACEs for Authenticated will apply.

We’ll need to have a way for a user to prove who they are to the satisfaction of the system.

The most common way of handling this is through a username and password.

A person provides both in an html form.

When the form is submitted, the system seeks a user with that name, and compares the passwords.

If there is no such user, or the password does not match, authentication fails.

Let’s imagine that Alice wants to authenticate with our website.

Her username is alice and her password is s3cr3t.

She fills these out in a form on our website and submits the form.

Our website looks for a User object in the database with the username alice.

Let’s imagine that there is one, so our site next compares the value she sent for her password to the value stored in the database.

If her stored password is also s3cr3t, then she is who she says she is.

All set, right?

The problem here is that the value we’ve stored for her password is in plain text.

This means that anyone could potentially steal our database and have access to all our users’ passwords.

Instead, we should encrypt her password with a strong one-way hash.

Then we can store the hashed value.

When she provides the plain text password to us, we encrypt it the same way, and compare the result to the stored value.

If they match, then we know the value she provided is the same we used to create the stored hash.

Python provides a number of libraries for implementing strong encryption.

You should always use a well-known library for encryption.

We’ll use a good one called Passlib.

This library provides a number of different algorithms and a context that implements a simple interface for each.

from passlib.context import CryptContext
password_context = CryptContext(schemes=['pbkdf2_sha512'])
hashed = password_context.encrypt('password')
if password_context.verify('password', hashed):
    print "It matched"

To install a new package as a dependency, we add the package to our list in setup.py.

Passlib provides a large number of different hashing schemes. Some (like bcrypt) require underlying C extensions to be compiled. If you do not have a C compiler, these extensions will be disabled.

requires = [
  ...
  'wtforms',
  'passlib',
]

Then, we re-install our package to pick up the new dependency:

(ljenv)$ python setup.py develop

note if you have a c compiler installed but not the Python dev headers, this may not work. Let me know if you get errors.

As noted above, the passlib library uses a context object to manage passwords.

This object supports a lot of functionality, but the only API we care about for this project is encrypting and verifying passwords.

We’ll create a single, global context to be used by our project.

Since the User class is the component in our system that should have the responsibility for password interactions, we’ll create our context in the same place it is defined.

In learning_journal/models.py add the following code:

# add an import at the top
from passlib.context import CryptContext

# then lower down, make a context at module scope:
password_context = CryptContext(schemes=['pbkdf2_sha512'])

Now that we have a context object available, let’s write an instance method for our User class that uses it to verify a plaintext password:

Again, in learning_journal/models.py add the following to the User class:

# add this method to the User class:
class User(Base):
    # ...
    def verify_password(self, password):
        return password_context.verify(password, self.password)

We’ll also need to have a user for our system.

We can use the database initialization script to create one for us.

Open learning_journal/scripts/initialzedb.py:

from learning_journal.models import password_context
from learning_journal.models import User
# and update the main function like so:
def main(argv=sys.argv):
    # ...
    with transaction.manager:
        # replace the code to create a MyModel instance
        encrypted = password_context.encrypt('admin')
        admin = User(name='admin', password=encrypted)
        DBSession.add(admin)

In order to get our user created, we’ll need to delete our database and re-build it.

Make sure you are in the folder where setup.py appears.

Then remove the sqlite database:

(ljenv)$ rm *.sqlite

And re-initialize:

(ljenv)$ initialize_learning_journal_db development.ini
...
2015-01-17 16:43:55,237 INFO  [sqlalchemy.engine.base.Engine][MainThread]
  INSERT INTO users (name, password) VALUES (?, ?)
2015-01-17 16:43:55,237 INFO  [sqlalchemy.engine.base.Engine][MainThread]
  ('admin', '$2a$10$4Z6RVNhTE21mPLJW5VeiVe0EG57gN/HOb7V7GUwIr4n1vE.wTTTzy')

Providing Login UI¶

We now have a user in our database with a strongly encrypted password.

We also have a method on our user model that will verify a supplied password against this encrypted version.

We must now provide a view that lets us log in to our application.

We start by adding a new route to our configuration in learning_journal/__init__.py:

config.add_rount('action' ...)
# ADD THIS
config.add_route('auth', '/sign/{action}', factory=EntryFactory)

It would be nice to use the form library again to make a login form.

Open learning_journal/forms.py and add the following:

# add an import:
from wtforms import PasswordField
# and a new form class
class LoginForm(Form):
    username = TextField(
        'Username', [validators.Length(min=1, max=255)]
    )
    password = PasswordField(
        'Password', [validators.Length(min=1, max=255)]
    )

Next, we’ll create a login view in learning_journal/views.py

# new imports:
from pyramid.security import forget, remember
from .forms import LoginForm
from .models import User
# and a new view
@view_config(route_name='auth', match_param='action=in', renderer='string',
     request_method='POST')
def sign_in(request):
    login_form = None
    if request.method == 'POST':
        login_form = LoginForm(request.POST)
    if login_form and login_form.validate():
        user = User.by_name(login_form.username.data)
        if user and user.verify_password(login_form.password.data):
            headers = remember(request, user.name)
        else:
            headers = forget(request)
    else:
        headers = forget(request)
    return HTTPFound(location=request.route_url('home'), headers=headers)

Notice that this view doesn’t render anything. No matter what, you end up returning to the home route.

We have to incorporate our login form somewhere.

The home page seems like a good place.

But we don’t want to show it all the time.

Only when we aren’t logged in already.

Let’s give that a whirl.

Pyramid security provides a method that returns the id of the user who is logged in, if any.

We can use that to update our home page in learning_journal/views.py:

# add an import:
from pyramid.security import authenticated_userid

# and update the index_page view:
@view_config(...)
def index_page(request):
    # ... get all entries here
    form = None
    if not authenticated_userid(request):
        form = LoginForm()
    return {'entries': entries, 'login_form': form}

Now we have to update the template for the index_page to display the form, if it is there

{% block body %}
{% if login_form %}
<aside><form action="{{ request.route_url('auth', action='in') }}" method="POST">
  {% for field in login_form %}
    {% if field.errors %}
      <ul>{% for error in field.errors %}
        <li>{{ error }}</li>
      {% endfor %}</ul>
    {% endif %}
      <p>{{ field.label }}: {{ field }}</p>
  {% endfor %}
  <p><input type="submit" name="Log In" value="Log In"/></p>
</form></aside>
{% endif %}
{% if entries %}
...

We should be ready at this point.

Fire up your application and see it in action:

(ljenv)$ pserve development.ini
Starting server in PID 84467.
serving on http://0.0.0.0:6543

Load the home page and see your login form:

http://localhost:6543/

Fill it in and submit the form, verify that you can add a new entry.

That’s enough for now. We have a working application.

When we return, we’ll deploy it.

Deploying An Application¶

Now that we have a working application, our next step is to deploy it.

This will allow us to interact with the application in a live setting.

We will be able to see the application from any computer, and can share it with friends and family.

To do this, we’ll be using one of the most popular platforms for deploying web applications today, Heroku.

Heroku¶

Heroku provides all the infrastructure needed to run many types of applications.

It also provides add-on services that support everything from analytics to payment processing.

Elaborate applications deployed on Heroku can be quite expensive.

But for simple applications like our learning journal, the price is just right: free

Heroku is predicated on interaction with a git repository.

You initialize a new Heroku app in a repository on your machine.

This adds Heroku as a remote to your repository.

When you are ready to deploy your application, you git push heroku master.

Adding a few special files to your repository allows Heroku to tell what kind of application you are creating.

It responds to your push by running an appropriate build process and then starting your app with a command you provide.

Preparing to Run Your App¶

In order for Heroku to deploy your application, it has to have a command it can run from a standard shell.

We could use the pserve command we’ve been using locally, but the server it uses is designed for development.

It’s not really suitable for a public deployment.

Instead we’ll use a more robust, production-ready server that came as one of our dependencies: waitress.

We’ll start by creating a python file that can be executed to start the waitress server.

At the very top level of your application project, in the same folder where you find setup.py, create a new file: runapp.py

import os
from paste.deploy import loadapp
from waitress import serve

if __name__ == "__main__":
    port = int(os.environ.get("PORT", 5000))
    app = loadapp('config:production.ini', relative_to='.')

    serve(app, host='0.0.0.0', port=port)

Once this exists, you can try running your app with it:

(ljenv)$ python runapp.py
serving on http://0.0.0.0:5000

This would be enough, but we also want to install our application as a Python package.

This will ensure that the dependencies for the application are installed.

Add a new file called simply run in the same folder:

#!/bin/bash
python setup.py develop
python runapp.py

The first line of this file will install our application and its dependencies.

The second line will execute the server script.

We’ll need to do the same thing for initializing the database.

Create another new file called build_db in the same folder:

#!/bin/bash
python setup.py develop
initialize_learning_journal_db production.ini

Now, add run, build_db and runapp.py to your repository and commit the changes.

For Heroku to use them, run and build_db must be executable

For OSX and Linux users this is easy (do the same for run and build_db):

(ljenv)$ chmod 755 run

Windows users, if you have git-bash, you can do the same

For the rest of you, try this (for both run and build_db):

C:\views\myproject>git ls-tree HEAD
...
100644 blob 55c0287d4ef21f15b97eb1f107451b88b479bffe    run
C:\views\myproject>git update-index --chmod=+x run
C:\views\myproject>git ls-tree HEAD
100755 blob 3689ebe2a18a1c8ec858cf531d8c0ec34c8405b4    run

Commit your changes to git to make them permanent.

Next, we have to inform Heroku that we will be using this script to run our application online

Heroku uses a special file called Procfile to do this.

Add that file now, in the same directory.

web: ./run

This file tells Heroku that we have one web process to run, and that it is the run script located right here.

Providing the ./ at the start of the file name allows the shell to execute scripts that are not on the system PATH.

Add this new file to your repository and commit it.

By default, Heroku uses the latest update of Python version 2.7 for any Python app.

You can override this and specify any runtime version of Python available in Heroku.

Just add a file called runtime.txt to your repository, with one line only:

python-3.5.0

Create that file, add it to your repository, and commit the changes.

Set Up a Heroku App¶

The next step is to create a new app with heroku.

You installed the Heroku toolbelt prior to class.

The toolbelt provides a command to create a new app.

From the root of your project (where the setup.py file is) run:

(ljenv)$ heroku create
Creating rocky-atoll-9934... done, stack is cedar-14
https://rocky-atoll-9934.herokuapp.com/ | https://git.heroku.com/rocky-atoll-9934.git
Git remote heroku added

Note that a new remote called heroku has been added:

$ git remote -v
heroku  https://git.heroku.com/rocky-atoll-9934.git (fetch)
heroku  https://git.heroku.com/rocky-atoll-9934.git (push)

Your application will require a database, but sqlite is not really appropriate for production.

For the deployed app, you’ll use PostgreSQL, the best open-source database.

Heroku provides an add-on that supports PostgreSQL, and you’ll need to set it up.

Again, use the Heroku Toolbelt:

$ heroku addons:create heroku-postgresql:hobby-dev
Creating postgresql-amorphous-6784... done, (free)
Adding postgresql-amorphous-6784 to rocky-atoll-9934... done
Setting DATABASE_URL and restarting rocky-atoll-9934... done, v3
Database has been created and is available
 ! This database is empty. If upgrading, you can transfer
 ! data from another database with pg:copy
Use `heroku addons:docs heroku-postgresql` to view documentation.

You can get information about the status of your PostgreSQL service with the toolbelt:

(ljenv)$ heroku pg
=== DATABASE_URL
Plan:        Hobby-dev
...
Data Size:   6.4 MB
Tables:      0
Rows:        0/10000 (In compliance)

And there is also information about the configuration for the database (and your app):

(ljenv)$ heroku config
=== rocky-atoll-9934 Config Vars
DATABASE_URL:                 postgres://<username>:<password>@<domain>:<port>/<database-name>

Configuration for Heroku¶

Notice that the configuration for our application on Heroku provides a specific database URL.

We could copy this value and paste it into our production.ini configuration file.

But if we do that, then we will be storing that value in GitHub, where anyone at all can see it.

That’s not particularly secure.

Luckily, Heroku provides configuration like the database URL in environment variables that we can read in Python.

In fact, we’ve already done this with our runapp.py script:

port = int(os.environ.get("PORT", 5000))

The Python standard library provides os.environ to allow access to environment variables from Python code.

This attribute is a dictionary keyed by the name of the variable.

We can use it to gain access to configuration provided by Heroku.

Update learning_journal/__init__.py like so:

# import the os module:
import os
# then look up the value we need for the database url
def main(global_config, **settings):
    # ...
    if 'DATABASE_URL' in os.environ:
        settings['sqlalchemy.url'] = os.environ['DATABASE_URL']
    engine = engine_from_config(settings, 'sqlalchemy.')
    # ...

We’ll need to make the same changes to learning_journal/scripts/initializedb.py:

def main(argv=sys.argv):
    # ...
    settings = get_appsettings(config_uri, options=options)
    if 'DATABASE_URL' in os.environ:
        settings['sqlalchemy.url'] = os.environ['DATABASE_URL']
    engine = engine_from_config(settings, 'sqlalchemy.')
    # ...

This mechanism allows us to defer other sensitive values such as the password for our initial user:

# in learning_journal/scripts/initializedb.py
with transaction.manager:
    password = os.environ.get('ADMIN_PASSWORD', 'admin')
    encrypted = password_context.encrypt(password)
    admin = User(name=u'admin', password=encrypted)
    DBSession.add(admin)

And for the secret value for our AuthTktAuthenticationPolicy

# in learning_journal/__init__.py
def main(global_config, **settings):
    # ...
    secret = os.environ.get('AUTH_SECRET', 'somesecret')
    ...
    authentication_policy=AuthTktAuthenticationPolicy(secret)
    # ...

We will now be looking for three values from the OS environment:

DATABASE_URL
ADMIN_PASSWORD
AUTH_SECRET

The DATABASE_URL value is set for us by the PosgreSQL add-on.

But the other two are not. We must set them ourselves using heroku config:set:

(ljenv)$ heroku config:set ADMIN_PASSWORD=<your password>
...
(ljenv)$ heroku config:set AUTH_SECRET=<a long random string>
...

You can see the values that you have set at any time using heroku config:

(ljenv)$ heroku config
=== rocky-atoll-9934 Config Vars
ADMIN_PASSWORD:               <your password>
AUTH_SECRET:                  <your auth secret value>
DATABASE_URL:                 <your db URL>

These values are sent and received using secure transport.

You do not need to worry about them being intercepted.

This mechanism allows you to place important configuration values outside the code for your application.

We’ve been handling our application’s dependencies by adding them to setup.py.

It’s a good idea to install all of these before attempting to run our app.

The pip package manager allows us to dump a list of the packages we’ve installed in a virtual environment using the freeze command:

(ljenv)$ pip freeze
...
zope.interface==4.1.3
zope.sqlalchemy==0.7.6

We can tell heroku to install these dependencies by creating a file called requirements.txt at the root of our project repository:

(ljenv)$ pip freeze > requirements.txt

Add this file to your repository and commit the changes.

But there is also a new dependency we’ve added that is only needed for Heroku.

Because we are using a PostgreSQL database, we need to install the psycopg2 package, which handles communicating with the database.

We don’t want to install this locally, though, where we use sqlite.

Go ahead and add one more line to requirements.txt with the latest version of the pyscopg2 package:

psycopg2==2.6.1

Commit the change to your repository.

Deployment¶

We are now ready to deploy our application.

All we need to do is push our repository to the heroku master:

(ljenv)$ git push heroku master
...
remote: Building source:
remote:
remote: -----> Python app detected
...
remote: Verifying deploy... done.
To https://git.heroku.com/rocky-atoll-9934.git
   b59b7c3..54f7e4d  master -> master

You can use the run command to execute arbitrary commands in the Heroku environment.

You can use this to initialize the database, using the shell script you created earlier:

(ljenv)$ heroku run ./build_db
...

This will install our application and then run the database initialization script.

At this point, you should be ready to view your application online.

Use the open command from heroku to open your website in a browser:

(ljenv)$ heroku open

If you don’t see your application, check to see if it is running:

(ljenv)$ heroku ps
=== web (1X): `./run`
web.1: up 2015/01/18 16:44:37 (~ 31m ago)

If you get no results, use the scale command to try turning on a web dyno:

(ljenv)$ heroku scale web=1
Scaling dynos... done, now running web at 1:1X.

Heroku pricing is dependent on the number of dynos you are running.

So long as you only run one dyno per application, you will remain in the free tier.

Scaling above one dyno will begin to incur costs.

Pay attention to the number of dynos you have running.

Troubleshooting problems with Heroku deployment can be challenging.

Your most powerful tool is the logs command:

(ljenv)$ heroku logs
...
2015-01-19T01:17:59.443720+00:00 app[web.1]: serving on http://0.0.0.0:53843
2015-01-19T01:17:59.505003+00:00 heroku[web.1]: State changed from starting to update

This command will print the last 50 or so lines of logging from your application.

You can use the -t flag to tail the logs.

This will continually update log entries to your terminal as you interact with the application.

Try logging in to your application with the password you set up in Heroku configuration.

Once you are logged in, try adding an entry or two.

You are now off to the races!

Congratulations

Adding Polish¶

So we have now deployed a running application.

But there are a number of things we can do to make the application better.

Let’s start by adding a way to log out.

Adding Logout¶

Our login view is already set up to work for logout.

What is the logical path taken if that view is accessed via GET?

All we need to do is add a view_config that allows that.

Open learning_journal/views.py and make these changes:

@view_config(route_name='auth', match_param='action=in', renderer='string',
         request_method='POST') # <-- THIS IS ALREADY THERE
# ADD THE FOLLOWING LINE
@view_config(route_name='auth', match_param='action=out', renderer='string')
# UPDATE THE VIEW FUNCTION NAME
def sign_in_out(request):
    # ...

The chief advantage of Heroku is that we can re-deploy with a single command.

Add and commit your changes to git.

Then re-deploy by pushing to the heroku master:

(ljenv)$ git push heroku master

Once that completes, you should be able to reload your application in the browser.

Visit the following URL path to test log out:

/sign/out

Hide UI for Anonymous¶

Another improvement we can make is to hide UI that is not available for users who are not logged in.

The first step is to update our detail view to tell us if someone is logged in:

# learning_journal/views.py
@view_config(route_name='detail', renderer='templates/detail.jinja2')
def view(request):
    # ...
    logged_in = authenticated_userid(request)
    return {'entry': entry, 'logged_in': logged_in}

The authenticated_userid function returns the id of the logged in user, if there is one, and None if there is not.

We can use that.

First we can hide the UI for creating a new entry:

Edit templates/list.jinja2:

{% extends "layout.jinja2" %}
{% block body %}
<!-- ... ADD THE IF TAGS BELOW -->
{% if not login_form %}
<p><a href="{{ request.route_url('action', action='create') }}">New Entry</a></p>
{% endif %}
{% endblock %}

This relies on the fact that the login form will only be present if there is not an authenticated user.

Next, we can hide the UI for editing an existing entry:

Edit templates/detail.jinja2:

{% extends "layout.jinja2" %}
{% block body %}
<!-- ... WRAP THE EDIT LINK -->
<p>
  <a href="{{ request.route_url('home') }}">Go Back</a>
  {% if logged_in %}
  ::
  <a href="{{ request.route_url('action', action='edit', _query=(('id',entry.id),)) }}">
    Edit Entry</a>
  {% endif %}
</p>
{% endblock %}

Format Entries¶

It would be nice if our journal entries could have HTML formatting.

We could write HTML by hand in the body field, but that’d be a pain.

Instead, let’s allow ourselves to write entries in Markdown, a popular markup syntax used by GitHub and many other websites.

Python provides several libraries that implement markdown formatting.

They will take text that contains markdown formatting and convert it to HTML.

Let’s use one.

The first step, is to pick a package and add it to our dependencies.

My recommendation is the markdown python library.

Open setup.py and add the package to the requires list:

requires = [
    # ...
    'cryptacular',
    'markdown', # <-- ADD THIS
    ]

We’ll test this locally first, so go ahead and re-install your app:

(ljenv)$ python setup.py develop
...
Finished processing dependencies for learning-journal==0.0

We’ve seen before how Jinja2 provides a number of filters for values when rendering templates.

A nice feature of the templating language is that it also allows you to create your own filters.

Remember the template syntax for a filter:

{{ value|filter(arg1, ..., argN) }}

A filter is simply a function that takes the value to the left of the | character as a first argument, and any supplied arguments as the second and beyond:

def filter(value, arg1, ..., argN):
    # do something to value here

Creating a markdown filter will allow us to convert plain text stored in the database to HTML at template rendering time.

Open learning_journal/views.py and add the following:

# add two imports:
from jinja2 import Markup
import markdown
# and a function
def render_markdown(content):
    output = Markup(markdown.markdown(content))
    return output

The Markup class from jinja2 marks a string with HTML tags as “safe”.

This prevents the tags from being escaped when they are rendered into a page.

In order for Jinja2 to be aware that our filter exists, we need to register it.

In Pyramid, we do this in configuration.

Open development.ini and edit it as follows:

[app:main]
...
jinja2.filters =
    markdown = learning_journal.views.render_markdown

This informs the main app that we wish to register a jinja2 filter.

We will call it markdown and it will be embodied by the function we just wrote.

To see the results of our work, we’ll need to use the filter in a template somewhere.

I suggest using it in the learning_journal/templates/detail.jinja2 template:

{% extends "layout.jinja2" %}
{% block body %}
<article>
  <!-- EDIT THIS LINE -->
  <p>{{ entry.body|markdown }}</p>
  <!-- -->
</article>
<p>
<!-- -->
{% endblock %}

Start up your application, and create an entry using valid markdown formatting:

(ljenv)$ pserve development.ini
Starting server in PID 84331.
serving on http://0.0.0.0:6543

Once you save your entry, you should be able to see it with actual formatting: headers, bulleted lists, links, and so on.

That makes quite a difference.

Go ahead and add the same filter registration to production.ini

Then commit your changes and redeploy:

(ljenv)$ git push heroku master

Syntax Highlighting¶

The purpose of this journal is to allow you to write entries about the things you learn in this class and elsewhere.

Markdown formatting allows for “preformatted” blocks of text like code samples.

But there is nothing in markdown that handles colorizing code.

Luckily, the markdown package allows for extensions, and one of these supports colorization.

It requires the pygments library

Let’s set this up next.

Again, we need to install our new dependency first.

Add the following to requires in setup.py:

requires = [
    # ...
    'markdown',
    'pygments', # <-- ADD THIS LINE
    ]

Then re-install your app to pick up the software:

(ljenv)$ python setup.py develop
...
Finished processing dependencies for learning-journal==0.0

The next step is to extend our markdown filter in learning_journal/views.py with this feature.

def render_markdown(content):
    output = Markup(
        markdown.markdown(
            content,
            extensions=['codehilite(pygments_style=colorful)', 'fenced_code']
        )
    )
    return output

Now, you’ll be able to make highlighted code blocks just like in GitHub:

```python
def foo(x, y):
    return x**y
```

Code highlighting works by putting HTML <span> tags with special CSS classes around bits of your code.

We need to generate and add the css to support this.

You can use the pygmentize command from pygments to generate the css.

Make sure you are in the directory with setup.py when you run this:

(ljenv)$ pygmentize -f html -S colorful -a .codehilite \
     >> learning_journal/static/styles.css

The styles will be printed to standard out.

The >> shell operator appends the output to the file named.

Go ahead and restart your application and see the difference a little style makes:

(ljenv)$ pserve development.ini
Starting server in PID 84331.
serving on http://0.0.0.0:6543

Try writing an entry with a little Python code in it.

Python is not the only language available.

Any syntax covered by pygments lexers is available, just use the shortname from a lexer to get that type of style highlighting.

When you’ve got this working as you wish, go ahead and deploy it.

Add and commit all the changes you’ve made.

Then push your results to the heroku master:

(ljenv)$ git push heroku master

Homework¶

That’s just about enough for now.

There’s no homework for you to submit this week. You’ve worked hard enough.

Take the week to review what we’ve done and make sure you have a solid understanding of it.

If you wish, play with HTML and CSS to make your journal more personalized.

However, in preparation for our work with Django next week, I’d like you to get started a bit ahead of time.

Please read and follow along with this basic intro to Django.

See You Then