Tuesday, April 19, 2011

WAAT (Web Analytics Automation Testing Framework) is alive!!!

[UPDATE]  Check related post here (http://essenceoftesting.blogspot.com/2011/04/waat-web-analytics-automation-testing.html)

I am very happy to announce that the first release of WAAT is available for general use.

WAAT is hosted on github (https://github.com/anandbagmar/WAAT)

WAAT v1.3 can be used to test *almost any type of Web Analytic solution. Tested with Google Analytics and Omniture. This is platform dependent.

Binaries:
You can either get the code base from git@github.com:anandbagmar/WAAT.git, or, get the binaries available in the dist folder.

Documentation:
Documentation for using WAAT is available in various different formats:
> WAAT Readme.docx
> WAAT Readme.doc
> WAAT Readme.pdf
> WAAT Readme.html

These files are available in the docs folder.

The documentation is also part of the binary file downloads.

I am looking forward for your usage and comments to make this better and more usable.

Saturday, April 16, 2011

WAAT release update

I am almost ready with my first public release of WAAT. Some finishing touches remaining which is causing the delay for this.

For those not aware, here is what WAAT is:
> WAAT stands for Web Analytics Automation Testing Framework
> Developed as a Java jar to be used in existing testing frameworks to do web analytics automation
> Phase 1: Implemented for Omniture using Omniture Debugger -> Status: Completed
> Phase 2: Can be used to test *almost any type of Web Analytic solution. Tested with Google Analytics and Omniture. This is platform dependent.
-> Status: In progress. Documentation to be updated.
> Phase 3: Make WAAT available for Ruby / .Net testing frameworks. -> Status: To be started.


Look at my earlier post for more details on WAAT.

Wednesday, April 13, 2011

Interesting webinar coming up ... "Where Exploration and Automation meet: Leveraging..."

There is a very interesting and informative webinar on how to utilize automated functional testing within your organization.

This is scheduled for Thursday, April 21.

See this link for more information.

Saturday, April 9, 2011

Agile QA Process

After doing testing on multiple Agile projects, I have come to realize certain aspects about the process and techniques that are common across projects. Some things I have learned along the way, some, by reflection on the mistakes / sub-optimal things that I did.

I have written and published my thoughts around the "Agile QA Process", more particularly what techniques can be used to test effectively in the Iterations. The pdf is available here for your reading. (http://www.slideshare.net/abagmar/agile-qa-process)

Note: A process is something that should be tweaked and customized based on the context you are in. The process mentioned in the document should be taken in the same light.

Friday, April 8, 2011

WAAT - Web Analytics Automation Testing Framework

[UPDATE]  See my post about how you can get WAAT here (http://essenceoftesting.blogspot.com/2011/04/waat-is-alive.html).

Problem statement:

On one of the projects that I worked on, I needed to test if Omniture reporting was done correctly.

The client relied a lot on Omniture reports to understand and determine the direction of their business. They have a bunch of Omniture tags reported for a lot of different actions on the site. Manual testing was the only way this functionality could be done verified. But given the huge number of tags, it was never possible to be sure that all tags were being reported correctly on a regular basis.

So I came up with a strategy to remove this pain-point.

Approach:
I created a framework in our existing automation framework to do Omniture testing. The intention of creating this framework was:
1. There is minimal impact on existing tests.
2. There should be no need to duplicate the tests just to do Omniture testing.
3. Should be easy to use (specify Omniture tags for various different actions, enable testing, etc.)

How it helped us?
1. We provided a huge and reliable safety net to the client and the development team by having Omniture testing automated.
2. Reduced the manual testing effort required for this type of testing, and instead got some bandwidth to focus on other areas.

Next Steps:
I am making this into a generic framework - a.k.a. WAAT - Web Analytics Automation Testing Framework to enable others doing Omniture testing to easily automate this functionality. This project will be hosted on github.

Phase 1 of this implementation will be for Omniture Debugger and input data provided in xml format. This framework will be available as a jar file.

Phase 2 also now complete includes support for any Web Analytic tool. I have tested this with Google Analytics as well as Omniture (NOT using Omniture Debugger). This uses a generic mechanism to capture packets from the network layer and processes them appropriately. Given this generic approach to work with any Web Analytic tool, the framework does become OS dependent.

Watch this space for more information (instructions, links to github, etc). Also, please do contact me with ideas / suggestions / comments about the same.

Tuesday, January 18, 2011

My article on "Future of Test Automation Tools and Infrastructure" featured in The Smart Techie magazine

My article on the Future of Test Automation Tools and Infrastructure is featured in "The Smart Techie" magazine. You can check it out here or download the pdf from here.

Wednesday, December 8, 2010

MEDIA ALERT: Live Web Event: The End of the Free Internet

Paul Jay, senior editor of The Real News Network, will moderate a virtual panel discussion promoting a dialogue for the technology community about the technological and legal ramifications of the WikiLeaks shutdown.
   
Topics to be discussed include:

    * What does the cut off of service to WikiLeaks mean for the future of the Internet? 

    * Will digital journalism be less protected?

    * Was WikiLeaks afforded procedural protections before its website and DNS entries were shut down? What process should be required?

    * The Internet is vulnerable to internal threats. What technical innovations are needed?

    * Can leaks ever be stopped? Is it worth the price?


Click here (http://bit.ly/ejXRNX) for more information.

Tuesday, November 23, 2010

Focus and Positivity is the key!

There are certain things I have observed quite a few times in various different situations while testing. These things seem more like a pattern than exceptions, and many a times demotivate me to do the right thing, for myself, and the organization.


What are these things?

  • Testing is considered as a step child - at times not given enough "listening ear" as required.
  • Testing usually seems lower in priority (compared to other issues faced).
  • Clients / organization seem OK with bugs / defects going live or, in other words, they are OK with pushing code in Production without testing it properly.
  • QA team is not involved in all stages of the life cycle - leading to disconnect between team members, invalid / incorrect assumptions, etc.
  • QA capacity is not adequate for the amount of work actually needed to be done.
  • Team members say - "This [issue] is not my responsibility. Talk to someone else."
  • Team members say - "This [issue] is not your [QA] responsibility. You don't need to provide suggestions / solutions for it. Let the appropriate owners think / talk about it."

Tips to remain focused on the right thing
  • Don't be hesitant to raise issues (time and again).
  • Identify and keep highlighting risks - in current situation, their implications and the potential mitigation points.
  • Keep doing what is right for the project / client.
  • Identify and engage with the "correct" people to ensure issues are understood, highlighted and prioritized.
  • Know what is in your control, and what is not. Strive to effect a change in what is in your control. For things out of your control, raise it with the relevant people, and then feel satisfied that you have done your part.
  • Be positive.
I strongly believe that the last 2 items in the above list are the most crucial to remaining focused and being successful in whatever one does ... in personal life and in professional life.


Saturday, November 20, 2010

Generic Test Automation Frameworks - Good or Bad?

Many a times I have been part of discussions that we should build a generic test automation framework which all other project teams can leverage. Also, another thought on this has been that building test automation framework is not really a QA job, but a developer job.

I have different thoughts on this.
  1. From a capability building perspective, if we build a generic framework, how is the QA team going to learn to do things from ground-up, and also make it better?
  2. From a business perspective, if I as an organization have a generic framework, then am I going to give it for free to the client? If no, then it is as good as saying I have built the automation framework as a standalone product - which is fine. But, if I am giving it free to the client, then what value has it brought to me as an organization by designing and building this framework, and then I just give it off free?
  3. Developers can build a great framework - it is another product for them. However, the users of this framework are QA. More often than not, this framework that is built by the developers will be difficult to use by the QAs.
  • I have seen an example of a client-project, where our developers and QAs built the framework together. However, only the developers were able to make any core framework changes because of its complexity. Also, when the time came to hand-over the framework to the client, it was the developer(s) who had to provide training for this - the QAs were not even involved in this, because ...
So to cut the answer short, I don't like the idea of generic test automation frameworks!

Monday, October 11, 2010

vodQA 2 - plan so hard for it, and its over so soon!

We were planning for vodQA 2 since about 6-7 weeks. But we suffered our first hiccup when I had to go to the US on a QA Coaching assignment for 3 weeks. Fortunately, my return was a few days before vodQA (7th October 2010).

It was a tough time being away ... constantly thinking about if all things are taken care off or not, if we are getting adequate help for the core organizers or not, what blockers are existing, and so on and on.

However, our core team here in ThoughtWorks office really slogged it out, at times pulling people by their ears :) to ensure we are on track.

All this meant that I didn't have to do much when I came back.

On THE day, just before the event though, we suffered another hiccup - our audio-visual vendor refused to come up till a good few guests had already arrived. So it was awkward start to the event ... but things settled down pretty fast and then the event did go through quite smoothly.

Pictures from vodQA 2 are available here

A lot of positives from the event:
1. More than 101 external guests arrived. Of these there were a significant number of people who had also attended our first vodQA event on June 10, 2010.
2. The talks were very interesting, and very apt to the changing dynamics of the industry.
3. In the initial feedback that we have gone through, people seem to be craving for such type of events - which is a very positive sign for us that we are doing something right, giving something to the community that will help them share, learn, grow and connect.
4. We incorporated the feedback received in our first vodQA to make this more tuned to what people wanted to experience.

A few people mentioned in the event - "this is our QA family, and we want to see it grow".

Finally, the event came, and went - leaving behind lot of good memories. I will post the presentation and the videos links as soon as they are posted.

Meanwhile, here are the topics that were covered in vodQA 2:

Speaker / topic list:

Full Length talks
1. Deepak Gole & Saager Mhatre - Sapna Solutions
Automated acceptance testing for iPhone

2.Parul Mody & Vijay Khatke - BMC Software
Cloud Testing

3. Ashwini Malthankar - ThoughtWorks
Effective use of Continuous integration by QA

4. Vijay and Supriya - ThoughtWorks
Test your service not your UI

5. Ananthapadmanabhan R - ThoughtWorks
Twist : Evolving test suites over test cases

Lightning Talks
1. Satish Agrawal - Persistent Systems
Leadership and Innovation in a commoditized industry

2. Anay Nayak - ThoughtWorks
Fluent Interfaces and Matches Fish bowl topic: DNA of Test Automation