WAAT for Ruby on its way

I have started work on creating a Ruby gem for WAAT. This is going to sit on top of the version created for Java. Hopefully will be able to get it out soon.

Watch this space for more information.

Test Trend Analyzer (TTA)

There are many tools and utilities that provide ways to do test result reporting and analysis of those results. However, I have not found a good, generic way of doing some Trend Analysis of those results. 

Why do I need to Trend Analysis of the test results?

Long(er) duration projects / enterprise products need to know the state of the quality of the product over time. One also may need to know various other metrics around the testing - like number of tests, pass %, failure %, etc. over time.

The reports I have seen are very good about analyzing the current test results. However, I have not really come across a good generic tool that can be used in most environments for the Test Trend Analysis over a period of time.

I am thinking about developing a tool which can address this space. I call this - the Test Trend Analyzer (TTA).

Here is what I think TTA should do:

Supports:

• Work with reports generated by common unit-test frameworks (jUnit, nUnit, TestNG, TestUnit, style of reports)
• Provides Web Service interface to upload results to TTA
• Test Results uploaded will be stored in db
• Will work on Windows and Linux

Dashboard:

• Creates default TTA dashboard
• Customizable TTA dashboard
• Dashboard will be accessible via the browser

My questions to you:

• Do you think this will help? 
• What other features would you like to see in TTA?
• What other type of support would you like to have in TTA?

WAAT and HTTPS

While most sites use http to report tags to the web analytic tool, there are some cases where http is disabled and all traffic is using https only.

In such cases, there may be a problem in using the generic solution provided by WAAT.

I did some research, analysis and experimentation and here are my findings:

1. jpcap captures raw packets. It does not differentiate about http / https
2. There is no problem in WAAT. All it does it matches packets based on patterns you specify in the tests.
3. Since the requests are https based, WAAT is not able to match the packets, unless you specify encrypted packet identifiers and encrypted data in the xml file. firebug / fiddler / ethereal / wireshark / charles / burp / etc. does something extra in this regard to decode the packet information and show the raw content in the browser / tool.

So the question is what can be done next?

1. If it is possible for you to get the configuration in the test environments changed to have the web analytics request sent out on http (maybe along with https) request, that can resolve the issue. Once in a while you can then verify manually if requests are going out on https.
2. You can use Omniture Debugger - but the limitation in your case is that it will be available for Omniture only and not the other web analytic tools.
3. You can extend the HttpSniffer class (,say HttpsSniffer), and provide implementation to decode the captured packets before doing the validation. However, note that this will be a expensive operation as you will be decoding all the captured packets on the network interfaces on your machine and the packet(s) of your interest will be fractional of those captured.