= Email Notifications for Branches =

Only subscribers get email notifications.  If the owner/author of the
branch wants to receive emails, then they need to subscribe to the
branch.  There is one specific case where the owner of the branch
receives emails without being subscribed, and that is when the
branch details were changed by someone who is not a member of the owner
team.

There are two situations where emails are sent out for branches:
 - when a user modifies the state of the branch object using the UI
 - when the branch scanner scans new revisions in the branches history


== Email Format ==

All branch notification emails use a standard email template.

The functions that handle the emailing of the branch email messages are
in the mailnotification module.

The email_branch_modified_notifications function loads the email
template (branch-modified.txt), and sets the subject based on the branch
details.  The function also adds the header X-Launchpad-Branch, and
populates the footer of the email with the branch details, the reason
why the user is getting the email, and a link that they can click on to
unsubscribe or edit their email notification settings.  The function
also sends the email to the list of recipients.

    >>> from zope.component import getUtility
    >>> from lp.code.enums import (
    ...     BranchSubscriptionNotificationLevel, BranchSubscriptionDiffSize,
    ...     CodeReviewNotificationLevel)
    >>> from lp.code.interfaces.branchlookup import IBranchLookup
    >>> from lp.code.mail.branch import BranchMailer
    >>> from lp.testing.mail_helpers import pop_notifications
    >>> branch = getUtility(IBranchLookup).getByUniqueName(
    ...     '~name12/firefox/main')
    >>> subscription = branch.subscribe(
    ...     branch.owner,
    ...     BranchSubscriptionNotificationLevel.FULL,
    ...     BranchSubscriptionDiffSize.WHOLEDIFF,
    ...     CodeReviewNotificationLevel.NOEMAIL, branch.owner)
    >>> BranchMailer.forRevision(branch, 1, 'foo@canonical.com',
    ...     'The contents.', None, 'Subject line').sendAll()

    >>> notifications = pop_notifications()
    >>> len(notifications)
    1
    >>> branch_notification = notifications[0]
    >>> print branch_notification['To']
    Sample Person <test@canonical.com>
    >>> print branch_notification['From']
    foo@canonical.com
    >>> print branch_notification['Subject']
    Subject line
    >>> print branch_notification['X-Launchpad-Project']
    firefox
    >>> print branch_notification['X-Launchpad-Branch']
    ~name12/firefox/main
    >>> print branch_notification['X-Launchpad-Message-Rationale']
    Subscriber
    >>> notification_body = branch_notification.get_payload(decode=True)
    >>> print notification_body #doctest: -NORMALIZE_WHITESPACE
    The contents.
    <BLANKLINE>
    --
    lp://dev/~name12/firefox/main
    http://code.launchpad.dev/~name12/firefox/main
    <BLANKLINE>
    You are subscribed to branch lp://dev/~name12/firefox/main.
    To unsubscribe from this branch go to http://code.launchpad.dev/~name12/firefox/main/+edit-subscription
    <BLANKLINE>
    >>> branch.unsubscribe(branch.owner, branch.owner)


== Subscriptions ==

When users subscribe to the branch, they specify which branch
modified events they want to receive email for.  This is one of the
following four options:
  * No email
  * Attribute notifications only
  * Revision notifications only
  * All notifications

If the user specifies that they are interested in receiving revision
notifications, then they can additionally specify a size limit for the
diff that is generated by comparing the new branch revision to the
previous one.  The size limit is one of the following:
  * No diff
  * 500 lines
  * 1000 lines
  * 5000 lines
  * Send the entire diff

    >>> from lp.registry.interfaces.person import IPersonSet
    >>> from lp.code.interfaces.branch import IBranch, IBranchSet
    >>> personset = getUtility(IPersonSet)

    >>> def subscribe_user_by_email(branch, email, level, size, level2):
    ...     subscriber = personset.getByEmail(email)
    ...     branch.subscribe(subscriber, level, size, level2, subscriber)

    >>> subscribe_user_by_email(branch, 'no-priv@canonical.com',
    ...     BranchSubscriptionNotificationLevel.NOEMAIL,
    ...     BranchSubscriptionDiffSize.NODIFF,
    ...     CodeReviewNotificationLevel.NOEMAIL)

    >>> subscribe_user_by_email(branch, 'test@canonical.com',
    ...     BranchSubscriptionNotificationLevel.ATTRIBUTEONLY,
    ...     BranchSubscriptionDiffSize.NODIFF,
    ...     CodeReviewNotificationLevel.NOEMAIL)

    >>> subscribe_user_by_email(branch, 'carlos@canonical.com',
    ...     BranchSubscriptionNotificationLevel.DIFFSONLY,
    ...     BranchSubscriptionDiffSize.NODIFF,
    ...     CodeReviewNotificationLevel.NOEMAIL)

    >>> subscribe_user_by_email(branch, 'jeff.waugh@ubuntulinux.com',
    ...     BranchSubscriptionNotificationLevel.DIFFSONLY,
    ...     BranchSubscriptionDiffSize.HALFKLINES,
    ...     CodeReviewNotificationLevel.NOEMAIL)

    >>> subscribe_user_by_email(branch, 'celso.providelo@canonical.com',
    ...     BranchSubscriptionNotificationLevel.DIFFSONLY,
    ...     BranchSubscriptionDiffSize.ONEKLINES,
    ...     CodeReviewNotificationLevel.NOEMAIL)

    >>> subscribe_user_by_email(branch, 'daf@canonical.com',
    ...     BranchSubscriptionNotificationLevel.DIFFSONLY,
    ...     BranchSubscriptionDiffSize.FIVEKLINES,
    ...     CodeReviewNotificationLevel.NOEMAIL)

    >>> subscribe_user_by_email(branch, 'mark@example.com',
    ...     BranchSubscriptionNotificationLevel.FULL,
    ...     BranchSubscriptionDiffSize.WHOLEDIFF,
    ...     CodeReviewNotificationLevel.NOEMAIL)

Team are subscribed in the same way.

    >>> def subscribe_team_by_name(branch, name, level, size, level2):
    ...     team = personset.getByName(name)
    ...     branch.subscribe(team, level, size, level2, team.teamowner)

    >>> subscribe_team_by_name(branch, 'launchpad',
    ...     BranchSubscriptionNotificationLevel.FULL,
    ...     BranchSubscriptionDiffSize.WHOLEDIFF,
    ...     CodeReviewNotificationLevel.NOEMAIL)

And to make sure we have them:

    >>> for subscription in branch.subscriptions:
    ...     print subscription.person.name
    no-priv
    name12
    carlos
    jdub
    cprov
    daf
    mark
    launchpad

The getNotificationRecipients method returns an instance of
NotificationRecipientSet (see doc/notification_recipient_set.txt).  The
NotificationRecipientSet is used to remember why the email recipients
are getting the emails.  The branch object adds all the branch
subscriptions to the NotificationRecipientSet and sets the reason to be
the subscription itself.  The subscription itself is passed through as
the reason, since not every subscriber gets every type of email.  The
filtering of the subscriptions are done in the notification handlers.
The header value is also set and sent as part of the message in the
email header X-Launchpad-Message-Rationale.  The
X-Launchpad-Message-Rationale header is added to email sent by launchpad
to allow email filtering.

    >>> recipients = branch.getNotificationRecipients()
    >>> interested_levels = (
    ...     BranchSubscriptionNotificationLevel.DIFFSONLY,
    ...     BranchSubscriptionNotificationLevel.FULL)
    >>> for email in recipients.getEmails():
    ...     subscription, header = recipients.getReason(email)
    ...     if subscription.notification_level in interested_levels:
    ...         print email, subscription.max_diff_lines.title, header
    carlos@canonical.com          Don't send diffs  Subscriber
    celso.providelo@canonical.com 1000 lines        Subscriber
    daf@canonical.com             5000 lines        Subscriber
    foo.bar@canonical.com         Send entire diff  Subscriber @launchpad
    jeff.waugh@ubuntulinux.com    500 lines         Subscriber
    mark@example.com                  Send entire diff  Subscriber


== Limiting the size of diff received by email ==

    # A helper function to print out the To header and
    # email body
    >>> def print_to_and_body(email):
    ...     attachment = ''
    ...     if email.is_multipart():
    ...         root = email.get_payload()
    ...         body = root[0].get_payload(decode=True)
    ...         if len(root) > 1:
    ...             attachment = '\n' + root[1].get_payload(decode=True)
    ...     else:
    ...         body = email.get_payload(decode=True)
    ...     print 'To: %s\n%s%s' % (email['To'], body, attachment)

We need to create some sufficiently large diffs to compare against.

    >>> diff = '\n'.join([str(value) for value in xrange(6000)])
    >>> message = 'Test message.\n'

Send the revision notifications.

    >>> BranchMailer.forRevision(
    ...     branch, 1234, 'no-reply@canonical.com', message, diff,
    ...     None).sendAll()
    >>> notifications = pop_notifications()
    >>> len(notifications)
    6
    >>> msg = notifications.pop(0)
    >>> print_to_and_body(msg)
    To: =?utf-8?q?Carlos_Perell=C3=B3_Mar=C3=ADn?= <carlos@canonical.com>
    Test message.
    <BLANKLINE>
    <BLANKLINE>
    --...

There are also some useful headers for filtering emails.

    >>> print msg['X-Launchpad-Branch']
    ~name12/firefox/main
    >>> print msg['X-Launchpad-Branch-Revision-Number']
    1234
    >>> print msg['X-Launchpad-Project']
    firefox
    >>> print_to_and_body(notifications.pop(0))
    To: Celso Providelo <celso.providelo@canonical.com>
    Test message.
    <BLANKLINE>
    The size of the diff (6000 lines) is larger than your specified limit of
    1000 lines...

    >>> print_to_and_body(notifications.pop(0))
    To: Dafydd Harries <daf@canonical.com>
    Test message.
    <BLANKLINE>
    The size of the diff (6000 lines) is larger than your specified limit of
    5000 lines...

Foo Bar is getting the email due to his membership in the Launchpad
developers team.  Since the email is due to a team, there is no
unsubscribe link.

    >>> print_to_and_body(notifications.pop(0))
    To: Foo Bar <foo.bar@canonical.com>
    Test message.
    <BLANKLINE>
    ...
    Your team Launchpad Developers is subscribed to branch
    lp://dev/~name12/firefox/main.
    To unsubscribe from this branch go to
    http://code.launchpad.dev/~name12/firefox/main/+edit-subscription
    <BLANKLINE>
    0 1 ... 5999...


    >>> print_to_and_body(notifications.pop(0))
    To: Jeff Waugh <jeff.waugh@ubuntulinux.com>
    Test message.
    <BLANKLINE>
    The size of the diff (6000 lines) is larger than your specified limit of
    500 lines...

Mark's unsubscription link is to his personal branch subscription.

    >>> print_to_and_body(notifications.pop(0))
    To: Mark Shuttleworth <mark@example.com>
    Test message.
    <BLANKLINE>
    ...
    To unsubscribe from this branch go to
    http://code.launchpad.dev/~name12/firefox/main/+edit-subscription
    0 1 ... 5999...

And just to be sure, lets create one with 800 lines.

    >>> diff = '\n'.join([str(value) for value in xrange(800)])
    >>> BranchMailer.forRevision(
    ...     branch, 1234, 'no-reply@canonical.com', message, diff,
    ...     None).sendAll()
    >>> notifications = pop_notifications()
    >>> len(notifications)
    6

Still just the log message for carlos:

    >>> print_to_and_body(notifications.pop(0))
    To: =?utf-8?q?Carlos_Perell=C3=B3_Mar=C3=ADn?= <carlos@canonical.com>
    Test message.
    <BLANKLINE>
    <BLANKLINE>
    --...

Diff for celso:

    >>> print_to_and_body(notifications.pop(0))
    To: Celso Providelo <celso.providelo@canonical.com>
    Test message.
    ...
    <BLANKLINE>
    0 1 ... 799...

Diff for daf:

    >>> print_to_and_body(notifications.pop(0))
    To: Dafydd Harries <daf@canonical.com>
    Test message.
    ...
    <BLANKLINE>
    0 1 ... 799...

Everything for Foo Bar:

    >>> print_to_and_body(notifications.pop(0))
    To: Foo Bar <foo.bar@canonical.com>
    Test message.
    ...
    <BLANKLINE>
    0 1 ... 799...

Limit hit for jeff:

    >>> print_to_and_body(notifications.pop(0))
    To: Jeff Waugh <jeff.waugh@ubuntulinux.com>
    Test message.
    <BLANKLINE>
    The size of the diff (800 lines) is larger than your specified limit of 500
    lines...

And everything for mark:

    >>> print_to_and_body(notifications.pop(0))
    To: Mark Shuttleworth <mark@example.com>
    Test message.
    ...
    <BLANKLINE>
    0 1 ... 799...

Unsubscribe everybody.

    >>> for subscription in branch.subscriptions:
    ...     branch.unsubscribe(subscription.person, subscription.person)
    >>> len(list(branch.subscriptions))
    0


== Group subscriptions ==

If a group is subscribed the emails are sent to the members of that team.
If an individual is also subscribed to the branch, then the setting
the individual specifies overrides any setting that they would receive
from a team registration.

If a team is registered, and that team has an email address assigned,
then that email address is used for the notifications.

    >>> subscribe_user_by_email(branch, 'david.allouche@canonical.com',
    ...     BranchSubscriptionNotificationLevel.DIFFSONLY,
    ...     BranchSubscriptionDiffSize.HALFKLINES,
    ...     CodeReviewNotificationLevel.NOEMAIL)

    >>> subscribe_team_by_name(branch, 'vcs-imports',
    ...     BranchSubscriptionNotificationLevel.DIFFSONLY,
    ...     BranchSubscriptionDiffSize.FIVEKLINES,
    ...     CodeReviewNotificationLevel.NOEMAIL)

The ubuntu-team has an email address supplied (support@ubuntu.com), so
that is used rather than the email addresses of the seven members.

    >>> subscribe_team_by_name(branch, 'ubuntu-team',
    ...     BranchSubscriptionNotificationLevel.DIFFSONLY,
    ...     BranchSubscriptionDiffSize.ONEKLINES,
    ...     CodeReviewNotificationLevel.NOEMAIL)

    >>> recipients = branch.getNotificationRecipients()
    >>> for email in recipients.getEmails():
    ...     subscription, header = recipients.getReason(email)
    ...     if subscription.notification_level in interested_levels:
    ...         print email, subscription.max_diff_lines.title, header
    david.allouche@canonical.com  500 lines   Subscriber
    foo.bar@canonical.com        5000 lines   Subscriber @vcs-imports
    robertc@robertcollins.net    5000 lines   Subscriber @vcs-imports
    support@ubuntu.com           1000 lines   Subscriber @ubuntu-team


== Attribute emails ==

    # Another helper function to print out the To, From and Subject headers
    # and the email body
    >>> def print_email_details(email):
    ...     body = email.get_payload(decode=True)
    ...     if email.get_param('charset') is not None:
    ...         body = body.decode(email.get_param('charset'))
    ...     else:
    ...         body = body.decode('iso-8859-1')
    ...     print u'To: %s\nFrom: %s\nSubject: %s\n%s' % (
    ...         email['To'], email['From'], email['Subject'], body)

It is the form infrastructure that fires off the ObjectModifedEvent,
so we'll fake that bit here.  The page tests will check the emails
sent.

Resubscribe our test user.

    >>> subscribe_user_by_email(branch, 'test@canonical.com',
    ...     BranchSubscriptionNotificationLevel.ATTRIBUTEONLY,
    ...     BranchSubscriptionDiffSize.NODIFF,
    ...     CodeReviewNotificationLevel.NOEMAIL)

    >>> from zope.event import notify
    >>> from lazr.lifecycle.event import ObjectModifiedEvent
    >>> from lazr.lifecycle.snapshot import Snapshot

    >>> login('test@canonical.com')
    >>> before_modification = Snapshot(branch, providing=IBranch)

    >>> branch.whiteboard = 'This is the new whiteboard'

Even though the branch notification emails don't use the field
names just now, we'll pass them through anyway

    >>> notify(ObjectModifiedEvent(branch,
    ...                               before_modification,
    ...                               ['whiteboard']))

    >>> notifications = pop_notifications()
    >>> len(notifications)
    1
    >>> print_email_details(notifications.pop())
    To: Sample Person <test@canonical.com>
    From: Sample Person <test@canonical.com>
    Subject: [Branch ~name12/firefox/main]
    Whiteboard changed to:
    <BLANKLINE>
    This is the new whiteboard
    <BLANKLINE>
    --
    lp://dev/~name12/firefox/main
    http://code.launchpad.dev/~name12/firefox/main
    <BLANKLINE>
    You are subscribed to branch lp://dev/~name12/firefox/main.
    To unsubscribe from this branch go to
    http://code.launchpad.dev/~name12/firefox/main/+edit-subscription


The fields that are currently tracked with the delta, and cause
an email to be sent out are:
  * name
  * title
  * summary
  * url
  * whiteboard
  * lifecycle_status

So, if all the UI fields are changed, you should get an email that looks
something like this:

    >>> branch = getUtility(IBranchLookup).getByUniqueName(
    ...     '~name12/firefox/main')

    >>> before_modification = Snapshot(branch, providing=IBranch)

    >>> branch.name = 'new-name'
    >>> branch.url = 'http://example.com/foo'
    >>> branch.whiteboard = 'This is a multiline whiteboard\n' \
    ...  'with a really long line that should invoke the splitting ' \
    ...  'algorithm in the mail wrapper to make sure that the line ' \
    ...  'is not too long'
    >>> from lp.code.enums import BranchLifecycleStatus
    >>> branch.lifecycle_status = BranchLifecycleStatus.EXPERIMENTAL
    >>> updated_fields = ['name','title','summary','url','whiteboard','lifecycle_status']
    >>> notify(ObjectModifiedEvent(
    ...     branch, before_modification, updated_fields))

    >>> notifications = pop_notifications()
    >>> len(notifications)
    1
    >>> print_email_details(notifications.pop())
    To: Sample Person <test@canonical.com>
    From: Sample Person <test@canonical.com>
    Subject: [Branch ~name12/firefox/new-name]
        Name: main => new-name
        Branch URL: http://bazaar.example.com/mozilla@arch.ubuntu.com/mozilla--MAIN--0 => http://example.com/foo
        Status: Development => Experimental
    <BLANKLINE>
    <BLANKLINE>
    Whiteboard changed to:
    <BLANKLINE>
    This is a multiline whiteboard
    with a really long line that should invoke the splitting algorithm in the
    mail wrapper to make sure that the line is not too long
    <BLANKLINE>
    --
    lp://dev/~name12/firefox/new-name
    http://code.launchpad.dev/~name12/firefox/new-name
    <BLANKLINE>
    You are subscribed to branch lp://dev/~name12/firefox/new-name.
    To unsubscribe from this branch go to
    http://code.launchpad.dev/~name12/firefox/new-name/+edit-subscription


== Unicode in emails ==

All the text fields of a branch are considered unicode, so the email
must also handle the unicode.

    >>> before_modification = Snapshot(branch, providing=IBranch)

    >>> branch.whiteboard = u'A new \ua000 summary'
    >>> updated_fields = ['whiteboard']
    >>> notify(ObjectModifiedEvent(
    ...     branch, before_modification, updated_fields))

    >>> notifications = pop_notifications()
    >>> len(notifications)
    1
    >>> email = notifications.pop()
    >>> email.get_payload(decode=True).decode('utf-8').splitlines()
    [u'Whiteboard changed to:',
     u'',
     u'A new \ua000 summary',
     u'',
     u'--',
     u'lp://dev/~name12/firefox/new-name',
     u'http://code.launchpad.dev/~name12/firefox/new-name',
     u'',
     u'You are subscribed to branch lp://dev/~name12/firefox/new-name.',
     u'To unsubscribe from this branch go to
     http://code.launchpad.dev/~name12/firefox/new-name/+edit-subscription']


== Modifications by users other than the branch owner ==

If another user modified some branch attributes, then an email is sent
to the branch owner.

    >>> branch = getUtility(IBranchLookup).getByUniqueName(
    ...     '~name12/gnome-terminal/main')

There are no subscribers to this branch.

    >>> len(list(branch.subscribers))
    0

Login as an admin user so we can alter the branch.

    >>> login('foo.bar@canonical.com')
    >>> before_modification = Snapshot(branch, providing=IBranch)

    >>> branch.whiteboard = (
    ...     'Please refrain from bad language in a public arena.')
    >>> updated_fields = ['whiteboard']
    >>> notify(ObjectModifiedEvent(
    ...     branch, before_modification, updated_fields))

    >>> notifications = pop_notifications()
    >>> len(notifications)
    1
    >>> print_email_details(notifications.pop())
    To: Sample Person <test@canonical.com>
    From: Foo Bar <foo.bar@canonical.com>
    Subject: [Branch ~name12/gnome-terminal/main]
    <BLANKLINE>
    Whiteboard changed to:
    <BLANKLINE>
    Please refrain from bad language in a public arena.
    <BLANKLINE>
    --
    lp://dev/~name12/gnome-terminal/main
    http://code.launchpad.dev/~name12/gnome-terminal/main
    <BLANKLINE>
    You are getting this email as you are the owner of the branch and someone
    has edited the details.