Google Summer of Code - Week 4

Having written the code for the Download Action and excluding the hyphenation dictionary from the build in this Bug 1175555, we have to move forward and push the code into the codebase. But unfortunately if we push the code and run testcases it will fail. In particular the Reftest (Picture Perfect) tests will fail. The reftest harness compares the display of two pages - actual and reference. If the bitmaps resulting from displaying the two files are identical, the test passes. If they differ, the test fails.

There are thousands of tests in the codebase (~13000 cases approx.). The full log can be found in this try run results. These are testcases that are failing if we exclude the hyphenation-dictionary from the build.

REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 2194
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-1.html != http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-1-notref.html | (LOAD ONLY)
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-1a.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 2194
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-5.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-5-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 2481
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-6.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-6-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 890
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-7.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-7-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 890
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-xmllang-1.xhtml == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 2194
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-xmllang-1.xhtml != http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-1-notref.html | (LOAD ONLY)
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-xmllang-1a.xhtml == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 2194
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-xmllang-5.xhtml == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-5-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 2481
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-xmllang-6.xhtml == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-6-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 890
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-xmllang-7.xhtml == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-7-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 890
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-xmllang-11a.xhtml == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 2194
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-xmllang-11b.xhtml == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 2194
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-xmllang-13a.xhtml == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 2194
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-xmllang-13b.xhtml == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 2194
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-af-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-af-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 4608
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-bg-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-bg-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 4013
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-ca-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-ca-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 3464
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-cy-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-cy-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 4380
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-da-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-da-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 4249
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-de-1901-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-de-1901-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 4436
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-de-1996-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-de-1996-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 4436
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-de-1901-1.html != http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-de-1996-1.html | (LOAD ONLY)
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-de-ch-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-de-ch-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 3944
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-eo-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-eo-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 4150
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-es-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-es-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 4671
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-et-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-et-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 5311
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-fi-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-fi-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 5584
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-fr-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-fr-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 3847
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-gl-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-gl-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 4712
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-hr-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-hr-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 4567
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-hsb-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-hsb-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 5094
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-hu-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-hu-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 5151
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-ia-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-ia-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 4480
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-is-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-is-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 3726
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-it-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-it-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 4223
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-kmr-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-kmr-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 3845
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-la-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-la-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 3686
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-lt-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-lt-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 3730
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-mn-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-mn-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 4457
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-nb-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-nb-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 6100
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-nl-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-nl-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 4504
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-nn-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-nn-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 5890
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-pl-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-pl-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 7727
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-pt-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-pt-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 4886
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-ru-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-ru-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 4807
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-sh-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-sh-1-ref.html | (LOAD ONLY), max difference: 181, number of differing pixels: 1668
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-sl-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-sl-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 4827
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-sr-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-sr-1-ref.html | (LOAD ONLY), max difference: 181, number of differing pixels: 1668
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-sv-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-sv-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 3893
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-sv-1.html != http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-sv-1-notref.html | (LOAD ONLY)
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-tr-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-tr-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 4149
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-uk-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-uk-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 4536
REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-transformed-1.html == http://10.0.2.2:8854/tests/layout/reftests/text/auto-hyphenation-transformed-1-ref.html | (LOAD ONLY), max difference: 255, number of differing pixels: 2753

Now as stated in this Bug 1276560, we have to package the hyphenation dictionary seperately.

The idea is to exclude the hyphenation dictionary from the build. But when we are running the “reftests” we have a temporary test profile directory in which we are dumping all the necessary files to support the reftests and stop them from failing. To break down the process a little further

Exclude the Hyphenation-Dictionary from the build.
While running reftests create a temporary profile directory “hyphenation” containing all the hyphenation-dictionary in it.
Push the temporary profile to the device - sdcard/tests/reftest/profile

In order to support this process we have to modify the following files

/testing/testsuite-targets.mk
/testing/mozharness/configs/android/androidarm_4_3.py
/testing/mozharness/configs/android/androidarm.py
/layout/tools/reftest/runreftest.py
/layout/tools/reftest/remotereftest.py
/layout/tools/reftest/mach_commands.py

After bundling and pushing it to the device the tests fail again. It seems that we missed something very fundamental. We have bundled all the necessary dictionary files for all the languages and successfully pushed it to device’s test profile directory and verified it’s presence in the directory - sdcard/tests/reftest/profile/hyphenation/files. Upon investigating further I discovered that the Gecko lacked the code to look for hyphenation dictionary in the test profile directory in the device. Now we have to include the code to make Gecko look for the hyphenation dictionary in the directory - sdcard/tests/reftest/profile/hyphenation/files. In order to bring that change we have to modify the nsHyphenationManager.cpp

nsCOMPtr<nsIFile> profileDir;
  rv = NS_GetSpecialDirectory(NS_APP_USER_PROFILE_LOCAL_50_DIR,
                                       getter_AddRefs(profileDir));
  if (NS_SUCCEEDED(rv)) {
      profileDir->AppendNative(NS_LITERAL_CSTRING("hyphenation"));
      LoadPatternListFromDir(profileDir);
  }

After introducing this code, Gecko looks for the hyphenation dictonaries from the appropriate directory and the reftests successfully passes. The sample results can be found in this try run

The above mentioned work is bundled as a sample changeset which can be found here

Now I am working on splitting the work into pieces and filing a bug for independent and dependent components and push the code to the codebase.

Share on: