Next, there was a very light glitch when scrolling in upscaled images. This was caused by how I implemented scaling and memory usage reduction. I was creating subimages first and then scaling these to fit their destination rectangle. I now proceed with scaling before selecting subimages.

As my update section was explaining, I give users more flexibility of when to manipulate images and use GDK’s bilinear interpolation to increase image quality or when to limit memory usage by a single preference variable.

Finally, this new patch should be considered the final version for bug 395260. It also could be considered a first step toward 372462 resolution. Joe is currently refactoring the whole nsThebesImage code for the Gecko 1.9.1 release and I would expect him to port my GDK manipulations to its future code.

Thin client users, enjoy a more stable Mozilla environment. Here’s a precompiled version of my patch. Feel free to try it and please report any weird behavior with image rendering.

Download my patched Firefox 3.0.1 build (x86-linux)
Download my patched Firefox 3.0.1 build (x86-win32)
Download the patch only

On the other hand, storing pixmaps in X might not be the best solution to optimize the speed of image rendering. Considering that the full pixmap needs to be transfered between the application and X, rendering the first visible frame will be slower. Also, using unlimited memory in X may steal the available ressources for other applications.

Firefox has a long history of known bugs related to pixmap storage in X. I’ve focused on bugs 296818 and 395260 during the last weeks with the unique goal of changing way Mozilla handles pixmaps. The most affected users of pixmap storage are those who use Firefox on thin clients. These usually have low available memory and can’t be used to store pixmaps. This simple page on a 128 Mbs RAM thin clients takes a while to load a few images and then causes a quick OOM kill before page rendering ends. One might say that showing multiple 5000×1220 images resized to 50×50 might not be common on the web but isn’t the web full of bad html coders ?

Frederico Mena Quintero has been very concerned about this behavior and integrated some modifications in Firefox to increase the quality of pixmap management. The infamous MOZ_DISABLE_IMAGE_OPTIMIZE environment variable has also been integrated in Mozilla’s sources to reduce memory used by pixmap in X. Unfortunately, none of the integrated patches changes the behavior whitout any quality issue in the overall browing experience.

While comparing Firefox to Opera and Konqueror, I noticed that only Opera seemed to effectively manage it’s memory and pixmap caching. These graphics roughly represent both application memory and X memory when rendering this simple test page.

	X Memory	App Memory
Firefox 3.0
Opera 9.27
Konqueror 3.5.9

Note : these graphics have been generated using these basic homemade scripts to give an overview of the total memory usage. The y scale represents the memory usage in bytes while the x scale is the sample number. appmem.py - memx.py (generate application data with appmem.sh)

From these graphics, we can conclude that there’s is a clear possibility to optimize memory consumption by pre-manipulating images before we send them to X. At least, this is what Opera seems to do and that this results in better memory management. Why wouldn’t Mozilla applications keep the uncompressed image data locally instead of pushing it to X ?

There are actually two situations to take into consideration to achieve memory optimization. First, Mozilla should send only visible portions of images to X, this forces to recompute the visible portion on every user operation, such as scrolling or window resizing, and send the new sub-images to X. Image scaling also needs to be taken into consideration. For example, why would we transfer the full data of a 1920×1200 image to X if it has been resized to 1024×768 ? The pixmap would simply be using memory ignored when rendering. As the Cairo developer Carl Worth pointed out, the advantage of transferring a full pixmap to X and let it resize an image would be to use the rendering extension that could be available in X. Doing so still requires to transfer a full pixmap between the X application client and the X server. The transfer itself is very bandwidth consuming when using a remote X connection, even on a 100Mbit network. For example, a 5000×1200 - 24 bits image represents around 46 Mbs of data to transfer and around 4.6 seconds of delay before we can render the first image. There is probably also an overhead when transferring images using local sockets but this would need to be verified by performing additional tests.

At first, I had implemented a basic downscaling algorithm and sub-image creation from an image’s original data. Since there are references to the GDK library in Thebes, I took the freedom to use available functions to manipulate raw image data. As this page explains it, GDK uses the RGBA format to represent a GdkPixbuf while Cairo respects the X server’s ARGB format. On a second thought, this results in the red and blue channel being inverted while manipulating images. There’s no reason why this would have consequences on the resulting image. Furthermore, GDK offers flexibility on the algorithms to scale images while the X server uses the nearest-neighbor interpolation algorithm. Hence, the gdk_pixbuf_new_subpixbuf() and gdk_pixbuf_scale_simple() functions offer the quickest way to easily pre-manipulate images in Thebes before the pixmaps are sent to X through the Cairo library. Moreover, the quality of the downscaled images is WAY better when using an algorithm different to the nearest-neighbor interpolation. The difference in Firefox is shown here :

Downscaled images in Firefox 3.0 using X’s nearest-neighbor interpolation: (current behavior)

Downscaled images using GDK’s bilinear interpolation:

Mozilla’s QA team has built the Talos performance testing project to verify that code modifications do not decrease the product quality.
FF3 built from source output
FF3 patched sources output

Update : After testing the patch in a few environments, I discovered that the scrolling quality could be sluggish when Firefox was used on a thin client with an application server having low resources. My first thought was that the transfer of raw data to the X server added an important overhead to the quickness of rendering but that hasn’t been the case in other environments. Since we can assume that a thin client’s backend should have the minimum ressources to support mallocs without using the swap area, there should be no visible difference of usage on a thin client. To give users control over image manipulation, I added the “browser.gdk_interpolation_threshold_percent” preference variable that allows values between 1-100 and has for effect to either use or bypass GDK image manipulations. The default value is 50% which only affects images being scaled by a factor of 50% or 200% and images with a visible portion under 50%. Lowering this value to it’s minimum (1%) would turn the feature off while using 100% will premanipulate any image overflowing it’s container. I’ve generated new Talos reports here run on a thin client. Finally, as Joe from #gfx did mention, I added support for images upscaling with GDK to increase the image quality by also using the bilinear interpolation algorithm. Here is the difference between an original 309×329 px image :
Upscaled image in Firefox 3.0 using X’s nearest-neighbor interpolation: (current behavior)

Upscale image using GDK’s bilinear interpolation:

Finally, here’s the complete patch that I hope will be integrated in Firefox’s and included in a future 3.0.x update. It currently only changes the behavior on UNIX systems since the main goal is to reduce memory consumption in X and that I am not aware of that kind of problems for GDI+ or Quartz backends.

diff : x-memory-optimization.diff
cvs diff : x-memory-optimization.cvs.diff

• Eclipse with the Eclipse-CDT platform
• Distcc - a distributed C/C++ compiler
• Mozilla Framework - the base framework used for Mozilla projects

Note: Mozilla has great documentation to help building any of their application. Please have a look there before posting any question. The #developers irc.mozilla.org channel is also a good start.

This tutorial assumes you’re using Ubuntu or a similar distro.

Build and configure Firefox

1. Install required tools and Firefox dependencies

sudo apt-get install cvs distcc eclipse eclipse-cdt libcurl4-openssl-dev distccmon-gnome
sudo apt-get build-dep firefox

2. Get the Firefox source

TAR :

wget ftp://ftp.mozilla.org/pub/mozilla.org/firefox/releases/3.0b5/source/firefox-3.0b5-source.tar.bz2
tar -xjf firefox-3.0b5-source.tar

CVS :

export CVSROOT=:pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot
cvs login
cvs -d :pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot co mozilla/client.mk
cd mozilla
make -f client.mk checkout

Note: Mozilla MXR offers a quick web search engine for source code

3. Create the Mozilla config file to build firefox

cd mozilla
echo “mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../obj-@CONFIG_GUESS@” >> .mozconfig
echo “ac_add_options –enable-application=browser” >> .mozconfig
echo “mk_add_options MOZ_CO_PROJECT=browser” >> .mozconfig
echo “ac_add_options –enable-debug” >> .mozconfig
echo “ac_add_options –disable-optimize” >> .mozconfig
echo “mk_add_options MOZ_MAKE_FLAGS=\”CC=’distcc gcc’ CXX=’distcc g++’ -j4\”" >> .mozconfig

Note: Add the last line only if you plan to use distcc. Also, replace the ‘j4′ parameter by the number of available processors (or cores) +2. Ex: if you have a total of 4 cores connected to your distcc server, use j6.

Install and configure distcc

1. Edit /etc/default/distcc

STARTDISTCC=”true”
ALLOWEDNETS=”10.145.0.0/16″
LISTENER=”0.0.0.0″

2. Add distcc hosts

mkdir ~/.distcc
echo “localhost build-machine” > ~/distcc/hosts
chmod 666 ~/.distcc/hosts

3. Restart the service

sudo /etc/init.d/distcc restart

Build Firefox

make -f client.mk build

Configure the Eclipse project

1. Create the project

File->New->Project…
C/C++ Project->Standard Make C++ Project
Project Name: Firefox
Location : (select the extracted mozilla folder)
Click “Finish”

2. Configure the project

Right-click on the new Firefox project in Eclipse’s left column
Click “Properties”
Select “Make C++ Project”
Change “Build command” to “make -f client.mk”
Change “Build (incremental build)” from “all” to “build”

3.  Build Firefox from Eclipse

You should remove the “Build automatically” option from the Project menu and build the application by right-clicking on the Firefox project and selecting “Build Project”
If you do not see the “Build Project” option, make sure you are using the C/C++ perspective.
This operation always takes a while since all subfolders and files need to be parsed (around 30k files)

Debugging Firefox

1. Create the Debug/Run configuration

Select Run->Debug… from the Eclipse menu
Right-click on C/C++ Local Application and select New
Project: firefox
C/C++ Application: (select browse and choose mozilla/../obj-i686-pc-linux-gnu/dist/bin/firefox-bin)

2. Add specific configuration

In the Arguments tab, change the working directory to mozilla/../obj-i686-pc-linux-gnu/dist/bin/

In the Environment tab, create two variables,

one with name LD_LIBRARY_PATH and value .:./plugins:.
one with name LIBRARY_PATH and value .:./components:.

In the Debugger tab, remove the checkbox on “Stop on startup at:”

3. Click Apply and hit Debug to start debugging Firefox. If you face any issue, you can try to switch the debugger from gdb/mi to gdb Debugger.

Now time to read Introduction to Mozilla Source Code. I might one day create a similar tutorial for a Visual Studio environment. Enjoy !

]]> http://www.francisrobichaud.com/index.php/2008/05/27/the-perfect-firefox-devenv/feed/ http://www.francisrobichaud.com/index.php/2008/02/26/apple-pro-package-what-actually-happened/ http://www.francisrobichaud.com/index.php/2008/02/26/apple-pro-package-what-actually-happened/#comments Tue, 26 Feb 2008 21:58:39 +0000 Skaber Apple Macbook Pro http://www.francisrobichaud.com/index.php/2008/02/26/apple-pro-package-what-actually-happened/ Apple did release complete Macbook Pro and Macbook revisions based on the Peryn architecture. Unfortunately, beside this expected update, LED for the 17″, larger drives, and the multi-touch trackpad, Apple decided to limit the quality of it’s product line to these minimal updates.

Apple used to have high-end laptops but the Peryn architecture was released months ago by competitors. It was quite normal to expect something more that caused this delay.

The iPhone firmware 1.1.4 was also released today without any clear informations on what it contains, we could expect this version to contain some traces of the upcoming SDK…

In conclusion, no Blu-ray, no SSD, no case redesign. It leaves me wondering why did I give any attention to this laptop. Conclusion: my Dell won’t be sold on eBay.

]]> http://www.francisrobichaud.com/index.php/2008/02/26/apple-pro-package-what-actually-happened/feed/