Security Testing - Types

Thinking about the different types of Security testings that we can do and also classifying them into the right buckets is very important.To summarise them Black box Testing ( Pen Testing)

• Little or no information is provided about the target

• Testing techniques start with looking for specific vulnerability signs but quickly moves into unscripted exploitation , trial & error

• Testing focusses on manipulating inputs and evaluating the responses

• A form of reverse engineering of exposed functionality

White box or Crystal box testing ( Not Pen Testing)

• Includes security focussed testing like - Source code reviews, authenticated vulnerability assessments & configuration audits

• More of a scripted test looking for specific items

Grey box Testing ( Optimised Pen testing

• Testing that uses black box techniques with greater visibility and/or access to the application to optimise testing

Read more

CasperJS + PhantomJS on Mac - First Test

1. Installation of CasperJS : open the Terminal and fire up these commands
$ brew update
$ brew install casperjs --devel

To test if the installation of casperjs was successful, type on terminal
$ which casperjs

you should get - /usr/local/bin/casperjs which means everything is fine
2. Install PhantomJS : open the terminal and fire up these commands
$ brew install node

This will install node. After which you can use the node/npm without using sudo
$ npm install -g phantomjs

To test if the installation of phantoms was successful, type on terminal
$ which phantomjs

you should get - /usr/local/bin/phantomjs which means everything is fine

Now you are all set to write your first test. Make sure you have BBEDIT installed ( best Code editor , i have seen ). Here, is a sample first test script for you

Remember to run this command in terminal, this will take care of your blank screenshots and https:// requests if any
$ casperjs --ignore-ssl-errors=yes --ssl-protocol=any caspertest.js

Read more

Cross Browser Playback in CodedUI

In effect , Cross Browser Playback is only useful to actually check UI differences between the different browsers. So the popular belief of replaying all the tests for testing the cross browser is just a myth. To improve your effectiveness, you may want to target specific tests at specific known UI problems in your app.   
Cross Browser Playback enables you to validate if your app is usable from different browsers. It also makes sense to create a few core end 2 end scenario’s you want to validate before you ship like purchasing an item from an online shop. You can also focus on critical business function that would seriously impact your business when stuff breaks. Thereby making playback resilient tests is crucial here so just ensure that your controls are  easy to identify, e.g. by ID across browsers

So, how do you get the ability to run the UI web tests you created in multiple browsers? First you need to have Visual Studio 2012 Update 1 or higher. So, this will not work with Visual Studio 2010 if you have not yet upgraded yet to the latest version of Visual Studio. The next thing you need to do is you need to go to the Visual Studio Gallery and there search for cross browser. Then you will find the Selenium components for Coded UI Cross Browser Testing. You can then download the installer, and then you need to install this package on every machine you want to play back the tests. So, when you have multiple test machines that are part of a test lab environment, for example in Team Foundation Server Lab Management, then you need to go to all these machines and install this package. You can also search for this package from the Visual Studio IDE. There you can go to the Tools menu, and there you go to the Extensions and Update menu. Here you can search the Visual Studio Gallery feed and then install straight from Visual Studio. Another thing of course that you need to install are the browsers Firefox and Chrome in order to play back on those browsers. One last thing to note is rather important, and that is that you can only record with Internet Explorer. So, if you choose to use the UI map files that we've discussed in the previous modules, then you can only record using Internet Explorer. You can still play back those recordings using the other browsers, but the recording itself needs to be done from IE.

Understanding cross Browser Playback

Look at the architecture of CodedUI. To understand how cross browser playback works we have to look at the bottom layer of this architectural diagram again. We know that CodedUI can work for any technology we'd like as long as there's a driver that can plug into the technology manager layer, and then we need to be able to select the right driver to run the test. Now, for cross browser playback what Microsoft did is write a switch in the web driver that can switch between the two technologies for playback. It still uses the standard implementation leveraging the MSHTML/DOM of Internet Explorer, but they now added the option to switch to a different engine called Selenium. Selenium is a technology solidly designed for browser testing. Selenium has the ability to play back scripts on different browsers for a few years now, and rather than building a competing technology, Microsoft adapted their engine to use the Selenium web driver to run the tests. You might ask yourself but what about Safari? I don't see that browser here in the playback browser symbols. Unfortunately that's true. There's no web driver in Selenium as well for supporting Safari, so that means that we can only play back on other WebKit-based browsers like Firefox and Chrome. This can give at least some confidence that it might work in the Apple WebKit-based browser, but unfortunately Google forked their implementation of WebKit for their browser so it becomes more likely each day that you will not find issues that might occur in Safari-based browsers because the browsers don't use theexact same rendering engine anymore.

How to Switch Browser on Playback
So, now we know what to install and how it works, but what do we need to do in our code to make this all work? The good news is almost nothing. The fact that Microsoft provides an implementation of their web driver in CodedUI that can switch technologies for playback makes switching browsers a breeze. The key element of making the switch is setting the current browser property of the BrowserWindow class. So, what we need to do is we need to specify the browser we want to use for playback. If we don't specify anything or IE, then this means it will be played back in Internet Explorer. If we set the current browser property to contain a text string Chrome before we call BrowserWindow.Launch, it will launch the Selenium Chrome web driver to run the test. If you specify the string Firefox, then it will use the default Selenium implementation that plays back on Firefox. One thing we of course also need to do is install the correct browsers on the machine, so we do need to install Google Chrome, Firefox, or Internet Explorer on the machine that runs the test.

Unsupported Features & Known Issues
So, there are a few caveats to look out for when using cross browser playback. Of course the thing we already discussed, we cannot play back on Safari-based browsers, and that's a problem we cannot fix other than by trying some of the key scenarios by hand and validating every now and then if you see differences in the browser behavior and watch out for those cases. The other problem that you might encounter is that search fails when it normally dependent on a filter to find the right control. A search is executed first based on the search properties, and when multiple controls are returned then a filter is applied to find the control in a set of returned controls. The cross browser implementation does not actually use the filter other than TagInstance, meaning that if your search relies on a filter on some property other than TagInstance of a control then the search will fail. To solve this problem you need to move the filter properties to the search properties. Since all the search properties are translated into a Selenium search, you will see that the search will then succeed. It's always best to try and use search as much as possible and try to keep away from filtering. But when using CodedUI record and playback, the filter properties are used more often, so therefore chances are that this will happen to you when you use record and playback, and it is less likely to happen when you hand code using the object model. Since search in Selenium is done in a different way, it is possible that you can get an error message Error Element does not exist in cache or that your search fails when an element appears delayed on the screen because JavaScript needs to complete on an AJAX call before it shows on the screen. In these cases there is a simple solution to fix this problem. The solution is to use the WaitForControlExist API and before we access any property on the control we can use the WaitForControlExist API to block the call until the control becomes available.

Read more

Writing your first test using Casperjs

Casperjs is a wonderful open source navigation, scripting and testing utility written in JS which uses PhantomJS webkit , which is a headless browser. It simplifies the process of defining a full navigation scenario and provides some high level methods for doing tasks such as – entering text, clicking, logging, navigation and logging events.

I had to spend some days to get my first test case running, so here I am making my notes which will benefit me in the longer run. ( In case you have more info, please leave a comment)

1. Download and set your environment paths for casperjs and phantomjs ( refer – previous post)

2. Create a folder say C:\JSplayground ( in my case) which will house your casperjs cases.

3. You can either use VS to write your new casperjs script or even a simple tool notepad++ will suffice.

4. Below is the code, I used, which you can also use to get your first case running. It’s pretty straight forward thing. You can copy the code and create a file with extension .js . In my case – sampletest.js

 

5. From the above code, If you have not figured out already, setting the capturePage as true or false drops the screen shot of the application under test in the same folder as your .js file. Ensure that the casper.userAgent is set to the same , as I was searching for long since some websites render and open differently for different browsers.

6. Once you have created your .js file, you can run now this script from your location by using the below command

 

7. I have given the option of using –ignore-ssl-errors=yes –ssl-protocol=any  so that even if your url is https:// your tests will run. BTB, Phantomjs had really bad support for SSL until 2.0 release. ( unfortunately I am on a newer version of Phantomjs and Casperjs is not equipped to handle 2.0. However, thanks to rdpanek , he has released a fix for CasperJS bootstrap/ Phantomjs 2.0 fix)

Read more

Phantomjs & Casperjs – Installation Simplified

To run Phantomjs & Casperjs on your system, here is the installation steps you need to follow. Though these are pretty simple ones, but the lack of proper documentation can actually send you on a leather hunt just to get your first case running. So here it is simplified.

1. Download Phantomjs from here.

2. Unzip the entire package to say c:\phantomjs . Ensure that the phantomjs.exe is copied from the subfolder bin to man folder. So your final folder should look somewhat like that.

3. Now head start –> right click my computer, choose properties –> Advanced system properties – > select advanced tab and click on Environment Variables.  Append ;C:\phantomjs to your PATH environment variable. Feel free to modify your installation and update the PATH settings to reflect the same

4. Now download Casperjs from their home page

5. Extract the contents to c:\casperjs

6. Ensure you add this PATH into your environment as shown above.

7. Restart system and try these two commands to make sure the PATHs are properly set.

phantomjs

You are now all set!

Read more

Create Multiple app.config files and run them with pre build events & msbuild

 

Today we were stumped with having different config files for automation for different environments like QA, Daily runs, BDT etc . While there are various ways, we could have resolved this, each of the methods we thought had one or the other challenge as we needed a single simple solution so that we could service 20 products line.

Also, making changes to the code base did not make sense. So here is how we achieved it

1.  Go to the solution – > right click choose configuration manager. In that screen use the drop down for active solution configuration and choose new. You will be prompted with the new solution configuration as shown below. We added DailyRun as our name.

2. Now go ahead and create any number of configuration files you want. Here in our case, we have BDT, DailyRun and Release1 as the configurations for different environment.

(side note : in our solution we had set the original app.config as a content and property always copy. The other configs were left out as it is. As app.config is the main file and that is the one which does all the hard work)

3.  Let's create a batch file called "copyalways.bat" and here's the contents:

Put this copyalways.bat file in the root of your project. Basically this batch file will copy a file over another if the files don't match.

4. Create a Pre-build Event. Right-click on your Project and select Properties. Click Build Events and in the "Pre-build event command line" and enter this value:


5. Now if you build, you'll see in the Build Output the batch file being run and the files being copied. Because it's a Pre-Build Event it'll be seen in both the Build Output in Visual Studio .NET.
 
And there you go. The connection string in the web.config now contains deployment-specific configuration data.
You can add only the parameter part to your build definition now. This will help you run the same solution with different config files.

(Note: we noticed that once the newer app.config is copied , the file was becoming read-only and was not taking in the next build config. The way to solve it would be to make it non read only within the code itself. I will post that code soon).
Done. Here is the complete code of the bat file

Read more

Thread.Sleep or Wait or FluentWait

Instead of sleep

public void clickOntaskType() {

getDriver().findElement(By.id("tasktypelink")).click();

sleep(500);

}

Prefer this fluentWait

public void clickOntaskType() {

getDriver().findElement(By.id("tasktypelink")).click();

SeleniumUtil.fluentWait(By.name("handle"), getDriver());

}

It’s more robust, deterministic, in case of element not found… the exception will be clearer.

Another alternative is

(new WebDriverWait(driver, 30)).until(new ExpectedCondition() {

public Boolean apply(WebDriver d) {

return d.getTitle().toLowerCase().startsWith("Awesome Tester");

Read more

Locating UI Elements–which to use

To locate an element we can use

the element’s ID

the element’s name attribute

an XPath statement

by a links text

document object model (DOM)

In practice, you will discover

id and name are often the easiest and sure way.

xpath are often brittle. for example you have 3 tables displayed but sometimes there are no data and the table isn’t rendered, your xpath locating the second table will not always return the intented one.

css are the way to go in conjunction of id and name !

locating links in an internationalized application is sometimes hard… try to use partial href.

Read more

Automated JS testing - 2

Now that we are here - Lets look at some interchangeable frameworks for JS testing and also see how we can translate the unit testing to an end to end testing

Example of JS unit testing framework - Used to test jQuery, JQuery UI and jQuery Mobile . QUnit can be used easily and also it can be used to test any generic js code itself . A Sample test is written below. You can use the cookbook to enhance your knowledge further 

test(“a test example”, function(){

ok( true, “this test passes”);

var value = “hello"

equal (value, “hello”, “We expect value to be hello”);

});

Read more

Automated Web Testing using Java Script - 1

I think, i have used this way to often and pleasantly surprised when i hear ppl mention - What? You can use JS for automation? Some even think i am crazy. But that’s ok. What purpose does it solve for me anyways!

Here, i intend to start a series of my experience and rants on using JS for automation . Before that, for the benefit of folks, let me list why JS for Testing ? 

• It is Free 
• It is open source
• It is modular
• It has an active and vibrant community
• If the client and servers use JS - why not tests?

Read more

Browser compatibility issues

Category	Issues	Root cause
CSS or Cascading Style Sheets  issues	Button alignment issues	Due to browser rendering behavior
	UI alignment issues	Due to browser rendering behavior
	Scrollbar issues on pop-up’s	Due to browser rendering behavior
	Issues due to absolute positioning or relative positioning of div. (Overlapping of base page and div)	Due to browser rendering behavior
	Dropdowns do not  work	Due to classes defined in style sheet not supported in different browsers
	Popups do  not display	Due to classes defined in style sheet not supported in different browsers
JQuery issues or JAVA script issues	Images on any buttons  do not appear	Due to classes not supported in different browsers
	Images on any panels  do not appear	Due to classes not supported in different browsers
	Java Script properties issues
	Dropdowns do not work	Due to classes defined in style sheet not supported in different browsers
	Popups do not display	Due to classes defined in style sheet not supported in different browsers
Common	Text wrapping issues in textboxes
	Resizing of windows  or resizing of textboxes
	Button functionality issues
	Tab order issues
	Browser Support for Graphic Formats Old and very old browsers do not support JPEGs PNG is an upcoming format supported by newer browsers
	Tables Cellspacing,  Cellpadding, and border  issues.
	Screen Variations screen size and settings issues
	Default Character Width Size Different platforms use a different default character width size. You'll notice that the same text and graphics will look slightly larger on a Mac than it does on a PC.   The two operating systems use a different default character width size. On the Mac, each character pixel is sized to be 1/72nd of an inch at the default 640x480 screen size. On the PC, each character pixel is sized to be 1/96th of an inch at the default 640x480 screen size. This means that a 72 pixel wide image will fill up one inch of Macintosh screen real estate—but not quite an inch of PC screen real estate
	Query string issues
	FAV icon issues
	Control ids are case sensitive in Mozilla.
	Inner HTML does not work in FF 1.      Inner HTML (getting) – returns the worst markup possible 2.      Inner HTML (setting) – doesn’t work on the elements you would want to dump a bunch of data into (e.g. tables and selects)
	A text node in Firefox allows only 4K data. So an XML Ajax response gets split up into multiple text child nodes instead of only one node. Its fine in Internet Explorer. For Firefox, to get the full data you either need to use node.normalize before you call node.firstChild or use node.textContent, both of which are Mozilla specific methods
	Internet Explorer does not replace   or HTML char code 160, you need to replace its Unicode equivalent \u00a0
	In Firefox a dynamically created input field inside a form (created using document.createElement) does not pass its value on form submit.
	document.getElementById in Internet Explorer will return an element even if the element name matches. Mozilla only returns element if id matches.
	In Internet Explorer if a select box has a value not represented by any of the options, it will display blank, Firefox displays the first option.

Read more

Firefox Vs IE compatibility issues

Below table depicts the common code changes that are required for FF compatibility in comparison with IE

IE Supported Code	FF Supported Code
Width, height gets inherited even if not specified	Need to specify width and Height otherwise doesn’t get inherited
Custom Properties can be directly accessed eg : .folderId	Custom Properties can be accessed only by getAttribute("property name') eg:  getAttribute('folderid')
.InnerText works in IE	Instead  use    .textContent
event.srcElement supported by IE only	event.target in FF also need to pass ‘event’ attribute explicitly  eg:  onclick="methodName(event);" -
SelectNodes() works in IE   SelectNodes() for paths starting with "\\pathName"	The following code needs to be used for SelectNodes() var xmlNodesFF = dom.getElementsByTagName(xPath);             for (var n = 0; n < xmlNodesFF.length; n++)             {                 var getXmlAttributes = dom.getElementsByTagName(xPath)[n].attributes;                 var selectXmlAttribute = getXmlAttributes.getNamedItem(attributeName).value;                 if (selectXmlAttribute == idValue)                 {                     selectNodesObj[i] = dom.getElementsByTagName(xPath)[n];                     i = i + 1;                 }                 else if (idValue == " " && attributeName == " ")// get all values                 {                     selectNodesObj[i] = dom.getElementsByTagName(xPath)[n];                     i = i + 1;                 } var i = 0;             var isMatch;             var xmlNodesFF = dom.getElementsByTagName(xPath);             for (var n = 0; n < xmlNodesFF.length; n++)             {                 var getXmlAttributes = dom.getElementsByTagName(xPath)[n].attributes;                 for (var cnt = 0; cnt < idValues.length; cnt++)                 {                     isMatch = false;                     var selectXmlAttribute = getXmlAttributes.getNamedItem(attributeNames[cnt]).value;                     if (selectXmlAttribute == idValues[cnt])                     {                         isMatch = true;                     }                     if (isMatch == false)                     {                         break;                     }                 }                 if (isMatch == true)                 {                     selectNodesObj[i] = dom.getElementsByTagName(xPath)[n];                     i = i + 1;                 }
Creating XML DOM object in IE is xmlDoc = new ActiveXObject("Microsoft.XMLDOM");	Creating XML DOM object in FF is xmlDoc  = document.implementation.createDocument("","doc",null);
to getattribute in IE xmlNode.getAttribute("Attribute Name");	In FF xmlNode.attributes["Attribute Name"].value;
.xml attributes is defined in IE	Need to use (new XMLSerializer()).serializeToString(xmlObject); to get the xml string
Setting Attribute : Node.setAttribute(attributeName) = value;	Setting  Attribute : Node.setAttribute(attributeName, attributeValue);
.Text works in IE	use .textContent to set and get text values
Width and height rendering differs in IE and FF because of the size	Width n height rendering differs in IE and FF because of the size
NodeTypeString exists in IE not in FF	NodeTypeString  undefined in FF use nodeType as it works for both IE and FF(nodeType = 1 defines ‘element’)
To loadXML from xml string as input , IE uses LoadXML(xmlstring)	FF uses var parser = new DOMParser()         xmlDoc = parser.parseFromString(xmlstring, "application/xml");

Read more