What's New : Added REST API and added tests with code coverage
A simple webhooks manager with event triggering.
Provides a web application interface and a REST API interface.
A simple event triggering endpoint and a consumer app are provided for testing.
Built using Django web framework and Huey task queue.
Additional developer setup notes
A Django webhooks application that:
- Re-queues the webhooks if the server doesn't respond with a 2xx code.
- Exposes a clean interface for registering and using the webhooks.
This Django server application triggers a set of events which executes registered webhooks for each event. The webhhok POST request execution will be submitted to a task queue which will do non-blocking execution with retries on failure.
The application allows registering of one or more webhook url's with one or more of these events.
This is similar to the github woobhooks registration for github events/actions.
This is a demo client application that can receive/consume a set of webhooks used for testing the server app.
Any of the public webhooks test sites like webhook.site can also be used but having a local consumer allows us to test failure cases and retry easily.
Consumer logs the request path and request body of the webhook POST request and responds with 200 OK and empty body.
$ python consumer/consumer.py
Starting webhook consumer at http://127.0.0.1:8888/
Received POST /alerts {"event": "create", "time": "2020-04-09T08:51:48.381993"}, Responded with 200
...You can also run the consumer to return Non-OK response code to test retry queueing.
$ python consumer/consumer.py 500
Starting webhook consumer at http://127.0.0.1:8888/
Received POST /alerts {"event": "create", "time": "2020-04-09T08:51:48.381993"}, Responded with 500
Received POST /alerts {"event": "create", "time": "2020-04-09T08:51:48.381993"}, Responded with 500
...Webhooks are created and linked to one or more of the event/s.
Events are triggered using a trigger API endpoint trigger/<event>
Can be triggered from the Web application browser or from command line.
$ curl http://localhost:8000/trigger/create
...
Triggered event "create" ...
...For each event trigger:
- Lookup the webhooks for the corresponding event from database
- Submit a POST request for each webhook with event details as the payload using a retrying task queue
- Any non-success response code or any exceptions will make the webhook and payload to be retried by the task queue
Task queue retry delay is configurable in
server/server/settings.pyasHOOKS_RETRY_DELAY_SECS = 3
Decided to use Huey task queue which is a lightweight alternative to more feature rich and popular task queues like Celery/RQ/Carrot.
- Can work with the built-in SQLite as the backend for demo/prototype
- Can switch to Redis as the backend easily for production
- Other task queues requires Redis or RabbitMQ even for prototype
- Has Django framework integration
The server application will use the built-in SQLite database with data access implemented through ORM layer using data models and QuerySets.
Dockerized as a simple single image
http://localhost:8000/api-help/ |
http://localhost:8000/api/ |
|---|---|
Run tests with coverage reporting
Available at http://localhost:8000/admin/
When deploying using Docker default admin credentials are admin/admin
- Dev
- Code quality - unit tests, docstrings, type hints, pylint etc.
- Add a common base template for all page templates
- Enable HTTPS for the server and for webhook consumer target URLs
- Enable user authentication for API and webapp
- Deploy (Changes for production deployment)
- Docker compose with separate images and shared volume for database
- Switch out SQLite with PostgreSQL/MySQL
- Use Redis backend for Huey or switch to advanced task queue like Celery
- Celery can provide dead-letter delivery, back-off retries etc.
- Deploy with uWSGI and NGINX