Tuesday, July 26, 2011

WAAT-Ruby - ready for use

WAAT-Ruby is now ready for use. 

Project hosted on github - http://github.com/anandbagmar/WAAT-Ruby
WAAT-Ruby gem available for download from here.
Documentation is available (on WAAt-Ruby wiki pages) here.

Since WAAT-Ruby uses WAAT-Java under the covers, I have kept the same version numbers for both platforms. The latest version is 1.4.

I have not yet pushed it out on rubygems.org. Will update once that is done.

So far I have tested this on the following environments:
  • Windows 7 - 64-bit with Ruby 1.8.6
  • RHEL 6 - 64-bit with Ruby 1.8.6 (I had difficulty in getting jpcap deployed on this environment). But once that was done, WAAT worked smoothly out of the box.
  • Ubuntu 10.x - 32-bit with Ruby 1.8.7
  • Ubuntu 10.x - 32-bit with Ruby 1.9.1
One important note:
If you are using WAAT (Java or Ruby) on any Linux-based environment, please note the Jpcap requirement for execution.
WAAT uses Jpcap for capturing network packets. Jpcap needs administrative privileges to do this work. See the Jpcap FAQs for more information about the same.


For all WAAT related blog posts, click here.

Wednesday, July 20, 2011

WAAT - Ruby .... are we there yet?

The WAAT ruby gem is almost ready. My colleagues are helping testing it out in different environments and am updating the documentation accordingly.

Once done, this will be available as a Ruby gem from WAAT-Ruby github project, and also from rubygems.org.

Contact me if you are interested in trying this out before release.

Thursday, July 14, 2011

What is your expiry date?

Recently when doing some online transaction using my credit card, something struck me ... I realized that the form asking for my credit card information was quite weird, and probably incorrect.

Here is a sample layout of what I am talking about:


Here, I am asked to enter the details in this order:
  • Credit card number 
  • Card holder's name
  • Expiry date 
  • And so on ...
As I was entering the information, I ended up questioning myself ... whose Expiry Date??? The card's or mine??? 

Simply based on the flow of information asked for, it is quite easy to associate the Expiry Date with the earlier field - the Card holder's name. Right?

Wouldn't the layout be better this way instead:
  • Card Holder name
  • Card number
  • Expiry date
  • CVV number

Or, another way can be:
  • Card number
  • Expiry date
  • CVV number
  • Card Holder name


I checked all my 10-15 (credit / debit / membership) cards that I have. All of them have the issue date / expiry date / validity period associated with the card number, and not the card holder's name.

This leads me to believe that no one did a usability check, or, in this context, shall we call it a reality check when designing the credit card form like the one shown above. 

I would have not let this design / layout get through. What would you do?

Thursday, July 7, 2011

Ruby Test Automation Framework Technology Stack Survey

WAAT  - Web Analytics Automation Testing Framework is currently available for java based Test Automation Frameworks. (http://essenceoftesting.blogspot.com/search/label/waat)

I am now working on making this available as a Ruby gem.

In order to support WAAT for a good-mix of test environments, I would like to understand the different environments and technology stacks that are typically used by teams in their Test Automation Framework.

Thank you for your time and help in providing this information.



Wednesday, July 6, 2011

RubyMine (and Cucumber) caching issue

I use RubyMine to write and implement my Cucumber features on Linux.
I have noticed one weird behavior at times.

Though my step definition is correct, and the test also runs fine, RubyMine flags the step as not implemented. For some reason, it is not able to find the corresponding implementation in the .rb step definition file.

On a haunch, I selected the "Invalidate Cache" in RubyMine's File menu, and selected the "Invalidate and Restart" option. Presto .... things started working properly again. 

Now I am wondering why did the RubyMine cache get messed up in the first place .....

Monday, June 27, 2011

WAAT for Ruby on its way

I have started work on creating a Ruby gem for WAAT. This is going to sit on top of the version created for Java. Hopefully will be able to get it out soon.

Watch this space for more information.

Friday, June 24, 2011

Test Trend Analyzer (TTA)

There are many tools and utilities that provide ways to do test result reporting and analysis of those results. However, I have not found a good, generic way of doing some Trend Analysis of those results. 


Why do I need to Trend Analysis of the test results?

Long(er) duration projects / enterprise products need to know the state of the quality of the product over time. One also may need to know various other metrics around the testing - like number of tests, pass %, failure %, etc. over time.

The reports I have seen are very good about analyzing the current test results. However, I have not really come across a good generic tool that can be used in most environments for the Test Trend Analysis over a period of time.

I am thinking about developing a tool which can address this space. I call this - the Test Trend Analyzer (TTA).

Here is what I think TTA should do:

Supports:
  • Work with reports generated by common unit-test frameworks (jUnit, nUnit, TestNG, TestUnit, style of reports)
  • Provides Web Service interface to upload results to TTA
  • Test Results uploaded will be stored in db
  • Will work on Windows and Linux

    Dashboard:
    • Creates default TTA dashboard
    • Customizable TTA dashboard
    • Dashboard will be accessible via the browser

      My questions to you:
      • Do you think this will help? 
      • What other features would you like to see in TTA?
      • What other type of support would you like to have in TTA?

      Tuesday, June 21, 2011

      WAAT and HTTPS

      While most sites use http to report tags to the web analytic tool, there are some cases where http is disabled and all traffic is using https only.

      In such cases, there may be a problem in using the generic solution provided by WAAT.

      I did some research, analysis and experimentation and here are my findings:
      1. jpcap captures raw packets. It does not differentiate about http / https
      2. There is no problem in WAAT. All it does it matches packets based on patterns you specify in the tests.
      3. Since the requests are https based, WAAT is not able to match the packets, unless you specify encrypted packet identifiers and encrypted data in the xml file. firebug / fiddler / ethereal / wireshark / charles / burp / etc. does something extra in this regard to decode the packet information and show the raw content in the browser / tool.

      So the question is what can be done next?
      1. If it is possible for you to get the configuration in the test environments changed to have the web analytics request sent out on http (maybe along with https) request, that can resolve the issue. Once in a while you can then verify manually if requests are going out on https.
      2. You can use Omniture Debugger - but the limitation in your case is that it will be available for Omniture only and not the other web analytic tools.
      3. You can extend the HttpSniffer class (,say HttpsSniffer), and provide implementation to decode the captured packets before doing the validation. However, note that this will be a expensive operation as you will be decoding all the captured packets on the network interfaces on your machine and the packet(s) of your interest will be fractional of those captured.

      Tuesday, April 19, 2011

      WAAT (Web Analytics Automation Testing Framework) is alive!!!

      [UPDATE]  Check related post here (http://essenceoftesting.blogspot.com/2011/04/waat-web-analytics-automation-testing.html)

      I am very happy to announce that the first release of WAAT is available for general use.

      WAAT is hosted on github (https://github.com/anandbagmar/WAAT)

      WAAT v1.3 can be used to test *almost any type of Web Analytic solution. Tested with Google Analytics and Omniture. This is platform dependent.

      Binaries:
      You can either get the code base from git@github.com:anandbagmar/WAAT.git, or, get the binaries available in the dist folder.

      Documentation:
      Documentation for using WAAT is available in various different formats:
      > WAAT Readme.docx
      > WAAT Readme.doc
      > WAAT Readme.pdf
      > WAAT Readme.html

      These files are available in the docs folder.

      The documentation is also part of the binary file downloads.

      I am looking forward for your usage and comments to make this better and more usable.