.. _aiohttp-demos-polls-getting-started: Getting started --------------- Let's start with basic folder structure: - project folder named ``polls``. A root of the project. Run all commands from here. - application folder named ``aiohttpdemo_polls`` inside of it - empty file ``main.py``. The place where web server will live We need this nested ``aiohttpdemo_polls`` so we can put config, tests and other related files next to it. It looks like this: .. code-block:: none polls <-- [current folder] └── aiohttpdemo_polls └── main.py aiohttp server is built around :class:`aiohttp.web.Application` instance. It is used for registering *startup*/*cleanup* signals, connecting routes etc. The following code creates an application:: # aiohttpdemo_polls/main.py from aiohttp import web app = web.Application() web.run_app(app) Save it and start server by running: .. code-block:: shell $ python aiohttpdemo_polls/main.py ======== Running on http://0.0.0.0:8080 ======== (Press CTRL+C to quit) Next, open the displayed link in a browser. It returns a ``404: Not Found`` error. To show something more meaningful than an error, let's create a route and a view. .. _aiohttp-demos-polls-views: Views ----- Let's start with the first views. Create the file ``aiohttpdemo_polls/views.py`` and add the following to it:: # aiohttpdemo_polls/views.py from aiohttp import web async def index(request): return web.Response(text='Hello Aiohttp!') This ``index`` view is the simplest view possible in Aiohttp. Now, we should create a route for this ``index`` view. Put the following into ``aiohttpdemo_polls/routes.py``. It is a good practice to separate views, routes, models etc. You'll have more of each file type, and it is nice to group them into different places:: # aiohttpdemo_polls/routes.py from views import index def setup_routes(app): app.router.add_get('/', index) We should add a call to the ``setup_routes`` function somewhere. The best place to do this is in ``main.py``:: # aiohttpdemo_polls/main.py from aiohttp import web from routes import setup_routes app = web.Application() setup_routes(app) web.run_app(app) Start server again using ``python aiohttpdemo_polls/main.py``. This time when we open the browser we see:: Hello Aiohttp! **Success!** Now, your working directory should look like this: .. code-block:: none . ├── .. └── polls └── aiohttpdemo_polls ├── main.py ├── routes.py └── views.py .. _aiohttp-demos-polls-configuration-files: Configuration files ------------------- .. note:: aiohttp is configuration agnostic. It means the library does not require any specific configuration approach, and it does not have built-in support for any config schema. Please note these facts: 1. 99% of servers have configuration files. 2. Most products (except Python-based solutions like Django and Flask) do not store configs with source code. For example Nginx has its own configuration files stored by default under ``/etc/nginx`` folder. MongoDB stores its config as ``/etc/mongodb.conf``. 3. Config file validation is a good idea. Strong checks may prevent unnecessary errors during product deployment. Thus, we **suggest** to use the following approach: 1. Push configs as ``yaml`` files (``json`` or ``ini`` is also good but ``yaml`` is preferred). 2. Load ``yaml`` config from a list of predefined locations, e.g. ``./config/app_cfg.yaml``, ``/etc/app_cfg.yaml``. 3. Keep the ability to override a config file by a command line parameter, e.g. ``./run_app --config=/opt/config/app_cfg.yaml``. 4. Apply strict validation checks to loaded dict. `trafaret `_, `colander `_ or `JSON schema `_ are good candidates for such job. One way to store your config is in folder at the same level as `aiohttpdemo_polls`. Create a ``config`` folder and config file at desired location. E.g.: .. code-block:: none . ├── .. └── polls <-- [BASE_DIR] │ ├── aiohttpdemo_polls │ ├── main.py │ ├── routes.py │ └── views.py │ └── config └── polls.yaml <-- [config file] Create a ``config/polls.yaml`` file with meaningful option names: .. code-block:: yaml # config/polls.yaml postgres: database: aiohttpdemo_polls user: aiohttpdemo_user password: aiohttpdemo_pass host: localhost port: 5432 minsize: 1 maxsize: 5 Install ``pyyaml`` package:: $ pip install pyyaml Let's also create a separate ``settings.py`` file. It helps to leave ``main.py`` clean and short:: # aiohttpdemo_polls/settings.py import pathlib import yaml BASE_DIR = pathlib.Path(__file__).parent.parent config_path = BASE_DIR / 'config' / 'polls.yaml' def get_config(path): with open(path) as f: config = yaml.safe_load(f) return config config = get_config(config_path) Next, load the config into the application: .. code-block:: python :emphasize-lines: 9 # aiohttpdemo_polls/main.py from aiohttp import web from settings import config from routes import setup_routes app = web.Application() setup_routes(app) app['config'] = config web.run_app(app) Now, try to run your app again. Make sure you are running it from ``BASE_DIR``:: $ python aiohttpdemo_polls/main.py ======== Running on http://0.0.0.0:8080 ======== (Press CTRL+C to quit) For the moment nothing should have changed in application's behavior. But at least we know how to configure our application. .. _aiohttp-demos-polls-database: Database -------- Server ^^^^^^ Here, we assume that you have running database and a user with write access. Refer to :ref:`aiohttp-demos-polls-preparations-database` for details. Schema ^^^^^^ We will use SQLAlchemy to describe database schema for two related models, ``question`` and ``choice``:: +---------------+ +---------------+ | question | | choice | +===============+ +===============+ | id | <---+ | id | +---------------+ | +---------------+ | question_text | | | choice_text | +---------------+ | +---------------+ | pub_date | | | votes | +---------------+ | +---------------+ +-------- | question_id | +---------------+ Create ``db.py`` file with database schemas:: # aiohttpdemo_polls/db.py from sqlalchemy import ( MetaData, Table, Column, ForeignKey, Integer, String, Date ) meta = MetaData() question = Table( 'question', meta, Column('id', Integer, primary_key=True), Column('question_text', String(200), nullable=False), Column('pub_date', Date, nullable=False) ) choice = Table( 'choice', meta, Column('id', Integer, primary_key=True), Column('choice_text', String(200), nullable=False), Column('votes', Integer, server_default="0", nullable=False), Column('question_id', Integer, ForeignKey('question.id', ondelete='CASCADE')) ) .. note:: It is possible to configure tables in a declarative style like so: .. code-block:: python class Question(Base): __tablename__ = 'question' id = Column(Integer, primary_key=True) question_text = Column(String(200), nullable=False) pub_date = Column(Date, nullable=False) But it doesn't give much benefits later on. SQLAlchemy ORM doesn't work in asynchronous style and as a result ``aiopg.sa`` doesn't support related ORM expressions such as ``Question.query.filter_by(question_text='Why').first()`` or ``session.query(TableName).all()``. You still can make ``select`` queries after some code modifications: .. code-block:: python from sqlalchemy.sql import select result = await conn.execute(select([Question])) instead of .. code-block:: python result = await conn.execute(question.select()) But it is not as easy to deal with as update/delete queries. Now we need to create tables in database as it was described with sqlalchemy. Helper script can do that for you. Create a new file ``init_db.py`` in project's root:: # polls/init_db.py from sqlalchemy import create_engine, MetaData from aiohttpdemo_polls.settings import config from aiohttpdemo_polls.db import question, choice DSN = "postgresql://{user}:{password}@{host}:{port}/{database}" def create_tables(engine): meta = MetaData() meta.create_all(bind=engine, tables=[question, choice]) def sample_data(engine): conn = engine.connect() conn.execute(question.insert(), [ {'question_text': 'What\'s new?', 'pub_date': '2015-12-15 17:17:49.629+02'} ]) conn.execute(choice.insert(), [ {'choice_text': 'Not much', 'votes': 0, 'question_id': 1}, {'choice_text': 'The sky', 'votes': 0, 'question_id': 1}, {'choice_text': 'Just hacking again', 'votes': 0, 'question_id': 1}, ]) conn.close() if __name__ == '__main__': db_url = DSN.format(**config['postgres']) engine = create_engine(db_url) create_tables(engine) sample_data(engine) .. note:: A more advanced version of this script is mentioned in :ref:`aiohttp-demos-polls-preparations-database` notes. Install the ``aiopg[sa]`` package (it will pull ``sqlalchemy`` alongside) to interact with the database, and run the script:: $ pip install aiopg[sa] $ python init_db.py .. note:: At this point we are not using any async features of the package. For this reason, you could have installed ``psycopg2`` package. Though since we are using sqlalchemy, we also could switch the type of database server. Now there should be one record for *question* with related *choice* options stored in corresponding tables in the database. Use ``psql``, ``pgAdmin`` or any other tool you like to check database contents: .. code-block:: text $ psql -U postgres -h localhost -p 5432 -d aiohttpdemo_polls aiohttpdemo_polls=# select * from question; id | question_text | pub_date ----+---------------+------------ 1 | What's new? | 2015-12-15 (1 row) Doing things at startup and shutdown ------------------------------------ Sometimes it is necessary to configure some component's setup and tear down. For a database this would be the creation of a connection or connection pool and closing it afterwards. Pieces of code below belong to ``aiohttpdemo_polls/db.py`` and ``aiohttpdemo_polls/main.py`` files. Complete files will be shown shortly after. .. _aiohttp-demos-polls-creating-connection-engine: Creating connection engine ^^^^^^^^^^^^^^^^^^^^^^^^^^ For making DB queries we need an engine instance. Assuming ``conf`` is a :class:`dict` with the configuration info for a Postgres connection, this could be done by the following async generator function: .. literalinclude:: ../demos/polls/aiohttpdemo_polls/db.py :pyobject: pg_context Add the code to ``aiohttpdemo_polls/db.py`` file. The best place for connecting to the DB is using the :attr:`~aiohtp.web.Application.cleanup_ctx` signal:: app.cleanup_ctx.append(pg_context) On startup, the code is run until the ``yield``. When the application is shutdown the code will resume and close the DB connection. .. note:: We could also have used separate startup/shutdown functions with the :attr:`~aiohtp.web.Application.on_startup` and :attr:`~aiohtp.web.Application.on_cleanup` signals. However, a cleanup context ties the 2 parts together so that the DB can be correctly shutdown even if an error occurs in another startup step. Complete files with changes ^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. code-block:: python :emphasize-lines: 33, 47 # aiohttpdemo_polls/db.py import aiopg.sa from sqlalchemy import ( MetaData, Table, Column, ForeignKey, Integer, String, Date ) __all__ = ['question', 'choice'] meta = MetaData() question = Table( 'question', meta, Column('id', Integer, primary_key=True), Column('question_text', String(200), nullable=False), Column('pub_date', Date, nullable=False) ) choice = Table( 'choice', meta, Column('id', Integer, primary_key=True), Column('choice_text', String(200), nullable=False), Column('votes', Integer, server_default="0", nullable=False), Column('question_id', Integer, ForeignKey('question.id', ondelete='CASCADE')) ) async def pg_context(app): conf = app['config']['postgres'] engine = await aiopg.sa.create_engine( database=conf['database'], user=conf['user'], password=conf['password'], host=conf['host'], port=conf['port'], minsize=conf['minsize'], maxsize=conf['maxsize'], ) app['db'] = engine yield app['db'].close() await app['db'].wait_closed() .. code-block:: python :emphasize-lines: 6, 11 # aiohttpdemo_polls/main.py from aiohttp import web from settings import config from routes import setup_routes from db import pg_context app = web.Application() app['config'] = config setup_routes(app) app.cleanup_ctx.append(pg_context) web.run_app(app) Since we now have database connection on start - let's use it! Modify index view: .. code-block:: python # aiohttpdemo_polls/views.py from aiohttp import web import db async def index(request): async with request.app['db'].acquire() as conn: cursor = await conn.execute(db.question.select()) records = await cursor.fetchall() questions = [dict(q) for q in records] return web.Response(text=str(questions)) Run server and you should get list of available questions (one record at the moment) with all fields. .. _aiohttp-demos-polls-templates: Templates --------- For setting up the template engine, we install the ``aiohttp_jinja2`` library first: .. code-block:: shell $ pip install aiohttp_jinja2 After installing, setup the library: .. code-block:: python :emphasize-lines: 3, 4, 12, 13 # aiohttpdemo_polls/main.py from aiohttp import web import aiohttp_jinja2 import jinja2 from settings import config, BASE_DIR from routes import setup_routes from db import pg_context app = web.Application() app['config'] = config aiohttp_jinja2.setup(app, loader=jinja2.FileSystemLoader(str(BASE_DIR / 'aiohttpdemo_polls' / 'templates'))) setup_routes(app) app.cleanup_ctx.append(pg_context) web.run_app(app) As you can see from setup above - templates should be placed at ``aiohttpdemo_polls/templates`` folder. Let's create simple template and modify index view to use it: .. code-block:: jinja {% set title = "Main" %} {% if questions %}
    {% for question in questions %}
  • {{ question.question_text }}
  • {% endfor %}
{% else %}

No questions are available.

{% endif %} Templates are a very convenient way for web page writing. If we return a dict with page content, the ``aiohttp_jinja2.template`` decorator processes the dict using the jinja2 template renderer. .. code-block:: python :emphasize-lines: 5, 11 # aiohttpdemo_polls/views.py import aiohttp_jinja2 import db @aiohttp_jinja2.template('index.html') async def index(request): async with request.app['db'].acquire() as conn: cursor = await conn.execute(db.question.select()) records = await cursor.fetchall() questions = [dict(q) for q in records] return {"questions": questions} Run the server and you should see a question decorated in html list element. Let's add more views: .. literalinclude:: ../demos/polls/aiohttpdemo_polls/views.py :pyobject: poll .. _aiohttp-demos-polls-static-files: Static files ------------ Any web site has static files such as: images, JavaScript sources, CSS files The best way to handle static files in production is by setting up a reverse proxy like NGINX or using CDN services. During development, handling static files using the aiohttp server is very convenient. Fortunately, this can be done easily by a single call: .. literalinclude:: ../demos/polls/aiohttpdemo_polls/routes.py :pyobject: setup_static_routes where ``project_root`` is the path to the root folder. .. _aiohttp-demos-polls-middlewares: Middlewares ----------- Middlewares are stacked around every web-handler. They are called before the handler for a pre-processing request. After getting a response back, they are used for post-processing the given response. A common use of middlewares is to implement custom error pages. Example from :ref:`aiohttp-web-middlewares` documentation will render 404 errors using a JSON response, as might be appropriate for a REST service. Here we'll create a little bit more complex middleware custom display pages for *404 Not Found* and *500 Internal Error*. Every middleware should accept two parameters, a *request* and a *handler*, and return the *response*. Middleware itself is a *coroutine* that can modify either request or response: Now, create a new ``middlewares.py`` file: .. literalinclude:: ../demos/polls/aiohttpdemo_polls/middlewares.py As you can see, we do nothing *before* the web handler. In the case of an ``HTTPException``, we use the Jinja2 template renderer based on ``ex.status`` *after* the request was handled. For other exceptions, we log the error and render our 500 template. Without the ``create_error_middleware`` function, the same task would take us many more ``if`` statements. We have registered middleware in ``app`` by adding it to ``app.middlewares``. Now, add a ``setup_middlewares`` step to the main file: .. code-block:: python :emphasize-lines: 6, 10 # aiohttpdemo_polls/main.py from aiohttp import web from settings import config from routes import setup_routes from middlewares import setup_middlewares app = web.Application() setup_routes(app) setup_middlewares(app) app['config'] = config web.run_app(app) Run the app again. To test, try an invalid url.