Speed up Selenium Tests through RAM Facts and Myths

I have been always curious if I could speed up my UI tests if the browser’s files (cache, profiles) are stored in RAM. When I was starting to test my idea, I was planning to put some majestic headline like “Speed up Your WebDriver Tests 120% Through RAM”. However, after a full day of hard research work, I found out that I was going to lie to you if I put a headline like this. For my benchmarks, I used all the major browsers- Chrome, Firefox, Internet Explorer and the new Edge. The tests were conducted against Hard Drive, SSD Drive, and RAM Drive. Find below all facts that break down the myth that you could make your browser fly if you put its files on an RAM drive.

<a href="http://i2.wp.com/automatetheplanet.com/wp-content/uploads/2015/08/speed-up-selenium-tests-ram-facts-myths.jpg" ="" title="" rel="prettyPhoto[main]">

So Far in the "WebDriver" Series

Prerequisites for the Research

1. Download and install an RAM Drive Software (for my tests I used a trial version of GiliSoft RAMDisk)

2. Create a virtual RAM Drive (at least 1 GB)

<a href="http://i2.wp.com/automatetheplanet.com/wp-content/uploads/2015/08/how-create-ramdisk.png" ="" title="" rel="prettyPhoto[main]">

Keep in mind that the program will take a real RAM of your machine so be sure that you have enough before proceeding with the experiments.

Benchmark Selenium Tests Execution Time- Browser’s Files Stored on Hard Drive

All quality performance tests should contain a baseline against the rest of the results are going to be compared. The baseline for this research will be the selenium tests execution time against browsers that store their files on the Hard Drive.

The following code is going to be used to measure the selenium tests speed against different browser configurations.

The test itself is straightforward- it only navigates to several URLs. In order WebDriver to wait for a full page load, the code uses the WaitUntilLoaded method. There the page’s document state is waited to be fully loaded though the WebDriver’s IJavaScriptExecuter.

Another fascinating piece of code that is used is the Profile method. First the garbage collector is called to clean up. After that, a stopwatch starts measuring the selenium tests execution time. The specified test is executed a predefined number of times (10). Finally, the results are stored in a file.

Firefox Driver Using Hard Drive

The initial version of the code is using FirefoxDriver. By the way by using the hard drive, I meant that the Firefox is storing all necessary files on the hard disk like cache files and user profiles.

The default folder for Windows is %APPDATA%\Mozilla\Firefox\Profiles\

Selenium Tests Execution Time- 56.112 seconds

Chrome Driver Using Hard Drive

Default Chrome user specific files location- C:\Users\%USERNAME%\AppData\Local\Google\Chrome\User Data\Default

To use the ChromeDriver, the TestInitialize should be changed a little bit. Also, you need to specify the location of the ChromeDriver.exe in the constructor.

Selenium Tests Execution Time- 29.174 seconds

Internet Explorer Driver Using Hard Drive

Default Internet Explorer cache locations:

Windows Vista and 7- %userprofile%\AppData\Local\Microsoft\Windows\Temporary Internet Files

Windows 8- %userprofile%\AppData\Local\Microsoft\Windows\INetCache

To use the InternetExplorerDriver, the TestInitialize should be changed. Also, you need to specify the location of the InternetExplorerDriver.exe in the constructor.

Additionally, the test should be modified to be able to pass. Through the usage of InternetExplorerOptions, the InternetExplorerDriver is capable of waiting the pages to be fully loaded. This means that we don’t need the WaitUntilLoaded method anymore.

Selenium Tests Execution Time- 36.324 seconds

Edge Driver Using Hard Drive

To use the InternetExplorerDriver, the TestInitialize should be changed. Furthermore, you can find out how to fully setup the Edge Driver in my article- Microsoft Edge WebDriver- What Everybody Ought to Know About.

Selenium Tests Execution Time- 14.197 seconds

Benchmark Selenium Tests Execution Time- Browser’s Files Stored on RAM Drive

The same code base is used for these series of tests. However, there is initial setup for every browser in order to move its files to the RAM Drive.

<a href="http://i2.wp.com/automatetheplanet.com/wp-content/uploads/2015/08/ram-drive-speed-up-tests.jpg" ="" title="" rel="prettyPhoto[main]">

Firefox Driver Using RAM Drive (Cache)

The first test that I performed was with Firefox, which cache files were moved to the RAM Drive.

Steps to Move Firefox Cache to RAM Drive

1. Type about:config into the location bar and press enter.

2. Accept the warning message that appears, you will be taken to a list of preferences.

<a href="http://i1.wp.com/automatetheplanet.com/wp-content/uploads/2015/08/about-config-firefox.png" ="" title="" rel="prettyPhoto[main]">

3. Right-click somewhere in the list and select “New > String”

<a href="http://i2.wp.com/automatetheplanet.com/wp-content/uploads/2015/08/create-new-string-about-config-firefox.png" ="" title="" rel="prettyPhoto[main]">

4. For the name of the preference type browser.cache.disk.parent_directory

5. For its value type the path to where you want to store the cache (path to the RAM Drive)

6. Next locate the preference browser.cache.disk.enable, it must be set to true, if it is not, double-click on it to change its value

<a href="http://i2.wp.com/automatetheplanet.com/wp-content/uploads/2015/08/browser.cache_.disk_.parent_directory.png" ="" title="" rel="prettyPhoto[main]">

Selenium Tests Execution Time- 54.713 seconds

Steps to Move Firefox Profiles to RAM Drive

The next phase of the speed up process is to move the Firefox profiles files to the RAM Drive.

1. If Firefox is open, close Firefox.

2. Press Windows +R on the keyboard. A Run dialog will open.

3. In the Run dialog box, type in firefox.exe -P

Note: You can use -P or -ProfileManager (either one should work).

<a href="http://i0.wp.com/automatetheplanet.com/wp-content/uploads/2015/08/firefox-profile-manager-run.png" ="" title="" rel="prettyPhoto[main]">

4. Click OK.

5. Create a new profile and sets its location to the RAM Drive

<a href="http://i1.wp.com/automatetheplanet.com/wp-content/uploads/2015/08/firefox-ram-profile.png" ="" title="" rel="prettyPhoto[main]">

Selenium Tests Execution Time- 56.856 seconds

Move Firefox Installation, Cache and Profiles to RAM Drive

1. Uninstall Firefox

2. Download it again

3. Choose installation location RAM Drive

<a href="http://i0.wp.com/automatetheplanet.com/wp-content/uploads/2015/08/best-result-from-all-measurements.jpg" ="" title="" rel="prettyPhoto[main]">

With this setup, there was a real improvement of almost 7%. It was the best that I achieved through the usage of the RAM Drive from all tested browsers.

Selenium Tests Execution Time- 48.970 seconds

Chrome Driver Using RAM Drive (Cache)

To set up Chrome to load its cache from the RAM Drive, follow the next steps.

1.  Press Windows+R on the keyboard. A Run dialog will open.

2. Type regedit

3. Locate the Registry key – HKEY_CLASSES_ROOT\ChromeHTML\shell\open\command

You should find a path to the Chrome executable there. All you need to do is to append the cache location and size to the path so that Chrome uses the right caching information when links are clicked, and Chrome is not open at that time.

4. Add –disk-cache-dir=”D:\Browsers-Caches\Chrome-Cache” after chrome.exe, so that it looks like the following:

“C:\Users\aangelov\AppData\Local\Google\Chrome\Application\chrome.exe” –disk-cache-dir=”D:\Browsers-Caches\Chrome-Cache”  — “%1″

<a href="http://i0.wp.com/automatetheplanet.com/wp-content/uploads/2015/08/regedit-chrome-cache-location.png" ="" title="" rel="prettyPhoto[main]">

The next time when you start Chrome it will load from your RAM Disk.

The execution of the tests against Chrome was the fastest when the cache is stored on the RAM. However, the improvement was only 1%.

Selenium Tests Execution Time- 28.077 seconds

Internet Explorer Driver Using RAM Drive (Cache)

Internet Explorer offers the easiest and most convenient way to change the default cache folder location.

1. Click Tools – Internet Options – under ‘Browsing History’ section

2. Choose ‘Settings’

3. Click ‘Move folder’ and provide the new location, e.g. Z:\IECache

After providing the new temp folder and clicking OK to save the new settings, Internet Explorer forcefully closes the current user session and logs the user off.

<a href="http://i2.wp.com/automatetheplanet.com/wp-content/uploads/2015/08/internet-explorer-change-cache-location.png" ="" title="" rel="prettyPhoto[main]">

Instead of speeding up the tests, they were executed even slower compared to the execution when the cached resources were present on the hard drive.

Selenium Tests Execution Time- 36.511 seconds

Benchmark Selenium Tests Execution Time- Browser’s Files Stored on SSD Drive

The setup for the tests against SSD drive is identical to the one against RAM Drive. So I will just share the test execution results.

<a href="http://i2.wp.com/automatetheplanet.com/wp-content/uploads/2015/08/ssd-drive-tests.jpg" ="" title="" rel="prettyPhoto[main]">

Firefox Driver Using SSD Drive (Cache, Profiles, Installation)

Selenium Tests Execution Time- 56.579 seconds

Chrome Driver Using SSD Drive

Selenium Tests Execution Time- 32.273 seconds

Internet Explorer Driver Using SSD Drive

Selenium Tests Execution Time- 33.287 seconds

Test Results Analysis

Below you can find a table with all execution times from the conducted experiments. The average execution time of all browsers is ~41 seconds, so the second column shows the improvement against it. The third column presents the advance of each test against the baseline for every browser that was the test execution on a hard drive.

<a href="http://i1.wp.com/automatetheplanet.com/wp-content/uploads/2015/08/test-execution-drives-table.png" ="" title="" rel="prettyPhoto[main]">

I think from the above results we can conclude that it is a myth that you can speed up your browser if you move some of its files to an RAM Drive. The same is valid for the SSD drives. As you can see from the picture, there was a real improvement with Chrome and Firefox. However, it is so small that I think it is safe to say that it doesn’t worth the time and resources to deal with it.

Source Code

You can download the full source code from my Github repository.

The post Speed up Selenium Tests through RAM Facts and Myths appeared first on Automate The Planet.