Development

To install for development:

  1. Fork the biweeklybudget repository on GitHub
  2. Create a new branch off of master in your fork.
$ virtualenv biweeklybudget
$ cd biweeklybudget && source bin/activate
$ pip install -e git+git@github.com:YOURNAME/biweeklybudget.git@BRANCHNAME#egg=biweeklybudget
$ cd src/biweeklybudget

The git clone you’re now in will probably be checked out to a specific commit, so you may want to git checkout BRANCHNAME.

Guidelines

  • pep8 compliant with some exceptions (see pytest.ini)
  • 100% test coverage with pytest (with valid tests)

Loading Data

The sample data used for acceptance tests is defined in biweeklybudget/tests/fixtures/sampledata.py. This data can be loaded by setting up the environment <_getting_started.setup> and then using the loaddata entrypoint (the following values for options are actually the defaults, but are shown for clarity):

loaddata -m biweeklybudget.tests.fixtures.sampledata -c SampleDataLoader

This entrypoint will drop all tables and data and then load fresh data from the specified class.

If you wish, you can copy biweeklybudget/tests/fixtures/sampledata.py to your customization package <_getting_started.customization> and edit it to load your own custom data. This should only be required if you plan on dropping and reinitializing the database often.

Testing

Testing is done via pytest, driven by tox.

  • testing is as simple as:
    • pip install tox
    • tox
  • If you want to pass additional arguments to pytest, add them to the tox command line after “–”. i.e., for verbose pytext output on py27 tests: tox -e py27 -- -v

For rapid iteration on tests, you can either use my toxit script to re-run the test commands in an existing tox environment, or you can use the bin/t and bin/ta scripts to run unit or acceptance tests, respectively, on only one module.

Unit Tests

There are minimal unit tests, really only some examples and room to test some potentially fragile code. Run them via the ^py\d+ tox environments.

Integration Tests

There’s a pytest marker for integration tests, effectively defined as anything that might use either a mocked/in-memory DB or the flask test client, but no HTTP server and no real RDBMS. Run them via the integration tox environment. But there aren’t any of them yet.

Acceptance Tests

There are acceptance tests, which use a real MySQL DB (see the connection string in tox.ini and conftest.py) and a real Flask HTTP server, and selenium. Run them via the acceptance tox environment.

The acceptance tests connect to a local MySQL database using a connection string specified by the DB_CONNSTRING environment variable, or defaulting to a DB name and user/password that can be seen in conftest.py. Once connected, the tests will drop all tables in the test DB, re-create all models/tables, and then load sample data. After the DB is initialized, tests will run the local Flask app on a random port, and run Selenium backed by PhantomJS.

If you want to run the acceptance tests without dumping and refreshing the test database, export the NO_REFRESH_DB environment variable. Setting the NO_CLASS_REFRESH_DB environment variable will prevent refreshing the DB after classes that manipulate data; this will cause subsequent tests to fail but can be useful for debugging.

Alembic DB Migrations

This project uses Alembic for DB migrations:

  • To generate migrations, run alembic -c biweeklybudget/alembic/alembic.ini revision --autogenerate -m "message" and examine/edit then commit the resulting file(s). This must be run before the model changes are applied to the DB. If adding new models, make sure to import the model class in models/__init__.py.
  • To apply migrations, run alembic -c biweeklybudget/alembic/alembic.ini upgrade head.
  • To see the current DB version, run alembic -c biweeklybudget/alembic/alembic.ini current.
  • To see migration history, run alembic -c biweeklybudget/alembic/alembic.ini history.

Database Debugging

If you set the SQL_ECHO environment variable to “true”, all SQL run by SQLAlchemy will be logged at INFO level.

To get an interactive Python shell with the database initialized, use python -i bin/db_tester.py.

Docker Image Build

Use the docker tox environment. See the docstring at the top of biweeklybudget/tests/docker_build.py for further information.

Frontend / UI

The UI is based on BlackrockDigital’s startbootstrap-sb-admin-2, currently as of the 3.3.7-1 GitHub release. It is currently not modified at all, but should it need to be rebuilt, this can be done with: pushd biweeklybudget/flaskapp/static/startbootstrap-sb-admin-2 && gulp

Sphinx also generates documentation for the custom javascript files. This must be done manually on a machine with jsdoc installed, via: tox -e jsdoc.

Release Checklist

  1. Open an issue for the release; cut a branch off master for that issue.
  2. Verify whether or not DB migrations are needed. If they are, ensure they’ve been created, tested and verified.
  3. Confirm that there are CHANGES.rst entries for all major changes.
  4. Rebuild documentation and javascript documentation locally: tox -e jsdoc,docs. Commit any changes.
  5. Run the Docker image build and tests locally: tox -e docker.
  6. Ensure that Travis tests passing in all environments.
  7. Ensure that test coverage is no less than the last release, and that there are acceptance tests for any non-trivial changes.
  8. If there have been any major visual or functional changes to the UI, regenerate screenshots via tox -e screenshots.
  9. Increment the version number in biweeklybudget/version.py and add version and release date to CHANGES.rst, then push to GitHub.
  10. Confirm that README.rst renders correctly on GitHub.
  11. Upload package to testpypi:
  1. Create a pull request for the release to be merged into master. Upon successful Travis build, merge it.
  2. Tag the release in Git, push tag to GitHub:
  • tag the release. for now the message is quite simple: git tag -a X.Y.Z -m 'X.Y.Z released YYYY-MM-DD'
  • push the tag to GitHub: git push origin X.Y.Z
  1. Upload package to live pypi:
    • twine upload dist/*
  2. Build and push the new Docker image:
  • Check out the git tag: git checkout X.Y.Z
  • Build the Docker image: DOCKER_BUILD_VER=X.Y.Z tox -e docker
  • Follow the instructions from that script to push the image to the Docker Hub and tag a “latest” version.
  1. make sure any GH issues fixed in the release were closed.
  2. Log in to readthedocs.org and enable building of the release tag. You may need to re-run another build to get the tag to be picked up.