~launchpad-pqm/launchpad/devel

= Searching Questions in Multiple Languages =

By default, only questions written in English or one of the user
preferred languages are listed and searched.


== Anonymous searching ==

For example, a user who isn't logged in will only see questions
written in English, in one of the language configured in his browser,
or inferred from GeoIP information. In this example, only English
questions will be shown when the user has not provided language
information.

    >>> anon_browser.open('http://launchpad.dev/distros/ubuntu/+questions')
    >>> soup = find_main_content(anon_browser.contents)
    >>> for question in soup.fetch('td', 'questionTITLE'):
    ...     print question.first('a').renderContents()
    Continue playing after shutdown
    Play DVDs in Totem
    mailto: problem in webpage
    Installation of Java Runtime Environment for Mozilla
    Slow system

    # Since we have more than 5 results, some of them are in the second batch.
    >>> anon_browser.getLink('Next').click()
    >>> soup = find_main_content(anon_browser.contents)
    >>> for question in soup.fetch('td', 'questionTITLE'):
    ...     print question.first('a').renderContents()
    Installation failed

The questions match the languages inferred from by GeoIP
127.0.0.1 is mapped to South Africa in the test suite. The language
control shows the intersection of the the user's languages and the
languages of the target's questions. In this example:
set(['af', 'en', 'st', 'xh', 'zu']) & set(['en', 'es']) == set(en).

    >>> sorted(anon_browser.getControl(name='field.language').options)
    ['en']

If the user unselects all the language options, then all questions,
whatever the language they are written in are displayed. Doing so,
the anonymous user will see a Spanish question.

    >>> anon_browser.getControl('English (en)').selected = False
    >>> anon_browser.getControl('Search', index=0).click()
    >>> table = find_tag_by_id(anon_browser.contents, 'question-listing')
    >>> for question in table.findAll('td', 'questionTITLE'):
    ...     question.first('a').renderContents()
    'Problema al recompilar kernel con soporte smp (doble-n\xc3\xbacleo)'
    'Continue playing after shutdown'
    'Play DVDs in Totem'
    'mailto: problem in webpage'
    'Installation of Java Runtime Environment for Mozilla'

While the user might recognize the first question above is in Spanish,
browsers and search engine robots need help. Each row in the list
of questions declares its language and text direction.

    >>> for question in table.findAll('tr', lang=True):
    ...     print 'lang="%s" dir="%s"' % (question['lang'], question['dir'])
    lang="es" dir="ltr"
    lang="en" dir="ltr"
    lang="en" dir="ltr"
    lang="en" dir="ltr"
    lang="en" dir="ltr"

Following the next link to the second page, the anonymous person or
robot will see the remaining questions. The last question is in
Arabic and is written from right-to-left.

    >>> anon_browser.getLink('Next').click()
    >>> table = find_tag_by_id(anon_browser.contents, 'question-listing')
    >>> for question in table.findAll('td', 'questionTITLE'):
    ...     question.first('a').renderContents()
    'Slow system'
    'Installation failed'
    '\xd8\xb9\xd9\x83\xd8\xb3 ...

    >>> for question in table.findAll('tr', lang=True):
    ...     print 'lang="%s" dir="%s"' % (question['lang'], question['dir'])
    lang="en" dir="ltr"
    lang="en" dir="ltr"
    lang="ar" dir="rtl"

When the project has no questions to search, we do not show the
language controls.

    >>> # Kubuntu must enable answers to access questions.
    >>> from zope.component import getUtility
    >>> from lp.app.enums import ServiceUsage
    >>> from lp.registry.interfaces.distribution import IDistributionSet

    >>> login('admin@canonical.com')
    >>> getUtility(IDistributionSet)['kubuntu'].answers_usage = (
    ...     ServiceUsage.LAUNCHPAD)
    >>> transaction.commit()
    >>> logout()

    >>> anon_browser.open('http://launchpad.dev/kubuntu/+questions')
    >>> anon_browser.getControl(name='field.language')
    Traceback (most recent call last):
      ...
    LookupError: name 'field.language...

    >>> content = find_main_content(anon_browser.contents).first('p')
    >>> print content.renderContents()
    There are no questions for Kubuntu with the requested statuses.

When the project has questions in only one language, and that language
is among the users' languages, the language controls are not displayed.
The mozilla-firefox sourcepackage only has English questions. When the
anonymous user makes a request from a GeoIP that has no languages
mapped, we assume he speaks the default language of English.

    >>> anon_browser.addHeader('X_FORWARDED_FOR', '172.16.1.1')
    >>> anon_browser.open(
    ...     'http://launchpad.dev/ubuntu/+source/mozilla-firefox/+questions')
    >>> anon_browser.getControl(name='field.language')
    Traceback (most recent call last):
      ...
    LookupError: name 'field.language...

But if the user configures his browser to accept Spanish and English
then questions with those language will be displayed:

    >>> anon_browser.addHeader('Accept-Language', 'es, en')
    >>> anon_browser.open('http://launchpad.dev/distros/ubuntu/+questions')
    >>> anon_browser.getControl('English (en)').selected
    True
    >>> anon_browser.getControl('Spanish (es)').selected
    True

    >>> soup = find_main_content(anon_browser.contents)
    >>> for question in soup.fetch('td', 'questionTITLE'):
    ...     question.first('a').renderContents()
    'Problema al recompilar kernel con soporte smp (doble-n\xc3\xbacleo)'
    'Continue playing after shutdown'
    'Play DVDs in Totem'
    'mailto: problem in webpage'
    'Installation of Java Runtime Environment for Mozilla'

    # Since we have more than 5 results, some of them are in the second batch.
    >>> anon_browser.getLink('Next').click()
    >>> soup = find_main_content(anon_browser.contents)
    >>> for question in soup.fetch('td', 'questionTITLE'):
    ...     question.first('a').renderContents()
    'Slow system'
    'Installation failed'


== Authenticated searching ==

Authenticated users without preferred languages are assumed to have
the languages determined by what their browser sends in the
Accept-Languages request, or by their GeoIP, just as with anonymous
users. In this example, No Privileges Person does not have preferred
languages, nor is his browser configured with a language; we use GeoIP
rules. As with the anonymous user, the intersection of the GeoIP
languages and the target's question languages is 'en'.

    >>> user_browser.open('http://launchpad.dev/distros/ubuntu/+questions')
    >>> sorted(user_browser.getControl(name='field.language').options)
    ['en']

When the project languages are just English, and the user speaks
that language, we do not show the language controls.

    >>> user_browser.addHeader('X_FORWARDED_FOR', '172.16.1.1')
    >>> user_browser.open(
    ...     'http://launchpad.dev/ubuntu/+source/mozilla-firefox/+questions')
    >>> user_browser.getControl(name='field.language')
    Traceback (most recent call last):
      ...
    LookupError: name 'field.language...

When No Privileges Person adds Spanish and English to his browser, they
are added to the language controls.

    >>> user_browser.addHeader('Accept-Language', 'es, en')
    >>> user_browser.open('http://launchpad.dev/distros/ubuntu/+questions')
    >>> sorted(user_browser.getControl(name='field.language').options)
    ['en', 'es']

Users that have configured their preferred languages may choose to
see questions for any, some, all, or none of their languages by
toggling the check box for each of their preferred languages. In
this example, Carlos speaks two languages (Spanish and Catalan),
and Answers adds English to the list of languages. There are no
Catalan questions, so no Catalan checkbox is displayed.

    >>> from lp.testing.pages import strip_label

    >>> browser.addHeader('Authorization', 'Basic carlos@canonical.com:test')
    >>> browser.open('http://launchpad.dev/distros/ubuntu/+questions')
    >>> language_control = browser.getControl(name='field.language')
    >>> for label in sorted(language_control.displayOptions):
    ...     strip_label(label)
    'English (en)'
    'Spanish (es)'
    >>> sorted(browser.getControl(name='field.language').options)
    ['en', 'es']

By unchecking a checkbox, Carlos can exclude English questions from
the search results.

    >>> browser.getControl('English (en)').selected = False
    >>> browser.getControl('Search', index=0).click()
    >>> content = find_main_content(browser.contents)
    >>> for question in content.fetch('td', 'questionTITLE'):
    ...     question.first('a').renderContents()
    'Problema al recompilar kernel con soporte smp (doble-n\xc3\xbacleo)'

Some users, translators in particular, speak an English variant.
English variants are considered to be English in the Answers,
so English is displayed among the languages. Dafydd is a translator
for en_GB, Japanese, and Welsh. knowing en_GB, he will be shown
English questions. As there are no Japanese or Welsh questions, there
will not be any controls present for those two languages when
searching Ubuntu.

    >>> daf_browser = setupBrowser(auth='Basic daf@canonical.com:daf')
    >>> daf_browser.open('http://launchpad.dev/~daf/+editlanguages')
    >>> daf_browser.getControl('English (United Kingdom)').selected
    True
    >>> daf_browser.getControl('Japanese').selected
    True
    >>> daf_browser.getControl('Welsh').selected
    True
    >>> daf_browser.getControl('English', index=1).selected
    False

The user's languages are presented as controls in the question form.
The controls are filters that allow the user to see questions in
his languages. Daf, in this example, can see a language filter for
English, and can use it to locate English questions.

    >>> daf_browser.open('http://launchpad.dev/distros/ubuntu/+questions')
    >>> language_control = daf_browser.getControl(name='field.language')
    >>> for label in language_control.displayOptions:
    ...     strip_label(label)
    'English (en)'

    >>> daf_browser.getControl(name='field.language').options
    ['en']

    >>> daf_browser.getControl('English (en)').selected = True
    >>> daf_browser.getControl('Search', index=0).click()
    >>> content = find_main_content(daf_browser.contents)
    >>> for question in content.fetch('td', 'questionTITLE'):
    ...     question.first('a').renderContents()
    'Continue playing after shutdown'
    'Play DVDs in Totem'
    'mailto: problem in webpage'
    'Installation of Java Runtime Environment for Mozilla'
    'Slow system'


== Questions by language ==

When open questions are in languages that no answer contacts
speak, the project questions page displays links to see those
questions. Each link is to a page that displays the questions
for a specific language. No Privileges Person is looking for a
project that needs help. He visits Kubuntu and does not see a need,
but when he visits Mozilla Firefox, he is informed that there are
questions going unanswered.

    >>> user_browser.open('http://answers.launchpad.dev/kubuntu')
    >>> paragraph = find_main_content(user_browser.contents).find('p')
    >>> print extract_text(paragraph)
    There are no questions for Kubuntu with the requested statuses.

    >>> user_browser.open('http://answers.launchpad.dev/firefox')
    >>> paragraph = find_main_content(user_browser.contents).find('p')
    >>> print extract_text(paragraph)
    Mozilla Firefox has unanswered questions in the following languages:
    1 in Portuguese (Brazil). Can you help?

No Privileges Person follows the 'Portuguese (Brazil)' link to view
questions by language. The language is predetermined by the view.
The user cannot change the language. The link presets the STATUS to
OPEN. No Privileges Person sees a single question as indicated by
the preceding page.

    >>> user_browser.getLink('Portuguese (Brazil)').click()
    >>> print user_browser.title
    Portuguese (Brazil) questions in Mozilla Firefox : Questions : Mozilla Firefox

    >>> language_field = user_browser.getControl(name='field.language')
    >>> print language_field.type
    hidden

    >>> labels = user_browser.getControl(name='field.status').displayValue
    >>> [strip_label(label) for label in labels]
    ['Open']

    >>> content = find_main_content(user_browser.contents)
    >>> for question in content.findAll('td', 'questionTITLE'):
    ...     question.first('a').renderContents()
    'Problemas de Impress\xc3\xa3o no Firefox'

The page in all other respects behaves like a question search page.


== My questions ignores preferred languages ==

The "My questions" view ignores the user's language preferences, because
he may change them over time, but he must always see his questions. For
example, Sample Person has asked a question in Arabic in the Ubuntu
project. He can see the question on the second page of "My questions"...

    >>> sample_person_browser = setupBrowser(
    ...     auth='Basic test@canonical.com:test')
    >>> sample_person_browser.open('http://answers.launchpad.dev/ubuntu')
    >>> sample_person_browser.getLink('My questions').click()
    >>> sample_person_browser.getLink('Next').click()
    >>> print sample_person_browser.title
    Questions you asked about Ubuntu : Questions : Ubuntu

    >>> questions = find_tag_by_id(
    ...     sample_person_browser.contents, 'question-listing')
    >>> for key in ('lang', 'dir'):
    ...     print '%s: %s ' % (key, questions.tbody.tr[key])
    lang: ar
    dir: rtl

    >>> for question in questions.findAll('td', {'class': 'questionTITLE'}):
    ...     question.find('a').renderContents()
    '\xd8\xb9\xd9\x83\xd8\xb3 \xd8\xa7\xd9\x84\xd8\xaa\xd8\xba...'

...even though he has not set Arabic as one of his preferred languages.

    >>> sample_person_browser.getLink(
    ...     'Change your preferred languages').click()
    >>> print sample_person_browser.title
    Language preferences...

    >>> sample_person_browser.getControl('Arabic', index=0).selected
    False