Here we introduce django-tidings by way of examples and discuss some theory behind its design.
A Simple Example¶
On support.mozilla.com, we host a wiki which houses documents in 80 different human languages. For each document, we keep a record of revisions (in the standard wiki fashion) stretching back to the document’s creation:
Document ---- Revision 1 \__ Revision 2 \__ Revision 3 \__ ...
We let users register their interest in (or watch) a specific language, and they are notified when any document in that language is edited. In our “edit page” view, we explicitly let the system know that a noteworthy event has occurred, like so…
revision’s document was written in English, sends a mail to
anyone who was watching English-language edits. The watching would have been
effected through view code like this:
def watch_language(request): """Start notifying the current user of edits in the request's language.""" EditInLanguageEvent.notify(request.user, language=request.locale) # ...and then render a page or something.
Thus we introduce the two core concepts of django-tidings:
- Things that occur, like the editing of a document in a certain language
- Subscriptions. Specifically, mappings from events to the users or email addresses which are interested in them
Everything in tidings centers around these two types of objects.
Events, Watches, and Scoping¶
django-tidings is basically a big dispatch engine: something happens (that is,
Event subclass fires), and tidings then has to
Watches are relevant so it knows
whom to mail. Each kind of event has an
event_type, an arbitrary string
that distinguishes it, and each watch references an event subclass by that
string. However, there is more to the watch-event relationship than that; a
watch has a number of other fields which can further refine its scope:
watch ---- event_type \__ content_type \__ object_id \__ 0..n key/value pairs ("filters")
In addition to an event type, a watch may also reference a content type, an
object ID, and one or more filters, key/value pairs whose values come out of
an enumerated set (no larger than integer space). The key concept in
django-tidings, the one which gives it its flexibility, is that only an Event
subclass determines the meaning of its Watches’ fields.
points to an Event subclass, but that is the only constant.
object_id are almost always used as their names imply—but only by
convention. And filters are designed from the start to be arbitrary.
As a user of django-tidings, you will be writing a lot of Event subclasses and
deciding how to make use of Watch’s fields for each. Let’s take apart our
simple example to see how the
EditInLanguageEvent class might be designed:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
class EditInLanguageEvent(Event): """Event fired when any document in a certain language is edited Takes a revision when constructed and filters according to that revision's document's language notify(), stop_notifying(), and is_notifying() take these args: (user_or_email, language=some_language) """ event_type = 'edited wiki document in language' filters = set(['language']) # for validation only def __init__(self, revision): super(EditInLanguageEvent, self).__init__() self.revision = revision def _users_watching(self, **kwargs): return self._users_watching_by_filter( language=self.revision.document.language, **kwargs) ...
This event makes use of only two
Watch fields: the
event_type (which is implicitly handled by the framework) and a filter with
the key “language”.
object_id are unused. The action
happens in the
_users_watching() method, which
Event.fire() calls to determine whom to mail. Line 20 calls
_users_watching_by_filter(), which is the most
interesting method in the entire framework. In essence, this line says “Find me
all the watches matching my
event_type and having a ‘language’ filter with
self.revision.document.language.” (It is always a good idea to
**kwargs along so you can support the
This is a good point to say a word about
WatchFilters. A filter is a key/value pair. The key is a
string and goes into the database verbatim. The value, however, is only a
4-byte unsigned int. If you pass a string as a watch filter value, it will be
hashed to make it fit. Thus, watch filters are no good for storing data but
only for distinguishing among members of enumerated sets.
An exception is if you pass an integer as a filter value. The framework will notice this and let the int through unmodified. Thus, you can put (unchecked) integer foreign key references into filters quite happily.
Details of the hashing behavior are documented in
Think back to our
It tells the framework to create a watch with the
document in locale' (tying it to
EditInLanguageEvent) and a filter
mapping “language” to some locale.
Now, what if we had made this call instead, omitting the
This says “
request.user is interested in every
regardless of language”, simply by omission of the “language” filter. A similar
logic applies to events which use the
leave them blank in a call to
notify(), and the
user will watch events with any value of them.
If, for some odd reason, a user ends up watching both all
EditInLanguageEvents and German
EditInLanguageEvents in particular,
never fear: he will not receive two mails every time someone edits a German
article. tidings will automatically de-duplicate users within the scope of one
event class. Also, when faced with a registered user and an anonymous
subscription having the same email address, tidings will favor the registered
user. That way, any mails you generate will have the opportunity to use a nice
Completing the Event Implementation¶
A few more methods are necessary to get to a fully working EditInLanguageEvent. Let’s add them now:
class EditInLanguageEvent(Event): # Previous methods here def _mails(self, users_and_watches): """Construct the mails to send.""" document = self.revision.document # This loop is shown for clarity, but in real code, you should use # the tidings.utils.emails_with_users_and_watches convenience # function. for user, watches in users_and_watches: yield EmailMessage( 'Notification: an edit!', 'Document %s was edited.' % document.title, settings.TIDINGS_FROM_ADDRESS, [user.email]) @classmethod def _activation_email(cls, watch, email): """Return an EmailMessage to send to anonymous watchers. They are expected to follow the activation URL sent in the email to activate their watch, so you should include at least that. """ return EmailMessage( 'Confirm your subscription', 'Click the link if you really want to subscribe: %s' % \ cls._activation_url(watch) settings.TIDINGS_FROM_ADDRESS, [email]) @classmethod def _activation_url(cls, watch): """Return a URL pointing to a view that activates the watch.""" return reverse('myapp.activate_watch', args=[watch.id, watch.secret])
Watching an Instance¶
Often, we want to watch for changes to a specific object rather than a class of
them. tidings comes with a purpose-built abstract superclass for this,
In the support.mozilla.com wiki, we allow a user to watch a specific document. For example…
With the help of
InstanceEvent, this event can be
implemented just by choosing an
event_type and a
because we need Revision info in addition to Document info when we build the
class EditDocumentEvent(InstanceEvent): """Event fired when a certain document is edited""" event_type = 'wiki edit document' content_type = Document def __init__(self, revision): """This is another common pattern: we need to pass the Document to InstanceEvent's constructor, but we also need to keep the new Revision around so we can pull info from it when building our mails.""" super(EditDocumentEvent, self).__init__(revision.document) self.revision = revision def _mails(self, users_and_watches): # ...
For more detail, see the
We have already established that mails get de-duplicated within the scope
of one event class, but what about across many events? What happens
when a document is edited and some user was watching both it specifically and
its language in general? Does he receive two mails? Not if you use
When your code does something that could cause both events to happen, the naive approach would be to call them serially:
That would send two mails. But if we use the magical
EventUnion construct instead…
…tidings is informed that you’re firing a bunch of events, and it sends only one mail.
A few notes:
The Container Pattern¶
One common case for de-duplication is when watchable objects contain other watchable objects, as in a discussion forum where users can watch both threads and entire forums:
forum ---- thread \__ thread \__ thread
In this case, we might imagine having a
NewPostInThreadEvent through which
users watch a thread and a
NewPostInForumEvent through which they watch a
whole forum. Both events would be
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
class NewPostInForumEvent(InstanceEvent): event_type = 'new post in forum' content_type = Forum def __init__(self, post): super(NewPostInForumEvent, self).__init__(post.thread.forum) # Need to store the post for _mails self.post = post class NewPostInThreadEvent(InstanceEvent): event_type = 'new post in thread' content_type = Thread def __init__(self, post): super(NewPostInThreadEvent, self).__init__(post.thread) # Need to store the post for _mails self.post = post def fire(self, **kwargs): """Notify not only watchers of this thread but of the parent forum as well.""" return EventUnion(self, NewPostInForumEvent(self.post)).fire(**kwargs) def _mails(self, users_and_watches): return emails_with_users_and_watches( 'New post: %s' % self.post.title, 'forums/email/new_post.ltxt', dict(post=post), users_and_watches)
On line 20, we cleverly override
fire(), replacing InstanceEvent’s simple
implementation with one that fires the union of both events. Thus, callers need
only ever fire
NewPostInThreadEvent, and it will take care of the rest.
NewPostInForumEvent will now be fired only from an
EventUnion (and not as the first argument), it can get
away without a
_mails implementation. The container pattern is very
slimming, both to callers and events.
Celery and Safe Asynchronous Tasks¶
Sending emails can be a slow process. By default,
Event.fire() uses Celery to process
the event asynchronously. The user’s request is faster, and the emails can
take as long as they need. This requires the pickle task serializer,
which has security concerns. Celery 3.1 is
the last version to enable pickle by default, and in Celery 4.0,
JSON is the default serializer.
You can avoid using
pickle by calling
This will process the event and send any emails, which could take a long time. You can move event processing to the backend by writing your own task:
from celery.task import task @task def fire_myevent(): MyEvent().fire(delay=False) # Process an event fire_myevent()
This can also be used with instance-based events, by loading the instance from the database inside of the task:
from celery.task import task from myapp.models import Instance @task def fire_myinstanceevent(instance_id): instance = Instance.objects.get(instance_id) MyInstance(instance).fire(delay=False) # Process an event fire_myinstanceevent(instance.id)
This will allow you to process events asynchronously, and to use safer serializers like the JSON serializer.