Evolution of vodQA - The Software Tester's Conference

Evolution of vodQA - The Software Tester's Conference!

Anand Bagmar

The birth of vodQA

Back in 2009 / 2010, I was looking to attend and speak in Testing Conferences, especially near, or close to my hometown, Pune.

My search results were very disappointing for the following reasons:

• I found that the conferences were not catered towards hands-on Testers.
• The conferences were catered for Leads / Managers/ Directors.
• The speakers spoke mostly about process, very high-level “things” related to Software Testing.
• There was a significant “cost” to attend the conferences – which meant that if one is really interested in attending to learn more, they had to think multiple times before doing so.

This was clearly not working for me, as well as other like-minded people I was interacting with.

Thanks to my colleagues in ThoughtWorks, we decided to do something about this.

In June 2010, we decided to start our own conference, hosted by ThoughtWorks, with the objective of “purely sharing our learning with the community, and also learn from the community”. This was the birth of “vodQA – value oriented discussion about Quality Analysis”.

To start anything, especially external / public facing, we needed to plan it well, and the first part of the planning is – what are we trying to achieve?

vodQA – Mission Statement

To make vodQA valuable to us, and other attendees, we came up with some guiding principles. These principles have kept on evolving over time like a natural process.

• vodQA is a practioners / hands-on conference.
• Anyone interested in the aspect and practice of Software Testing, regardless of role or organization, should be able to attend vodQA.
• Any topic related to Software Testing (manual / exploratory / techniques / tools / technologies / process).
• At ThoughtWorks, the Quality of the Software we build is of prime importance. To cater to that, we have a very strong Testing capability, skills and practices within the organization. Using vodQA as a platform, we want to share our learning with the Testing Community, so they can learn from what has worked well, or not so well, based on real experiences.
• Though at ThoughtWorks, we have “good” Testing practices, there are many more practices and experiences we can learn from the experiences of other organizations and individuals. The vodQA platform should able to provide new learning from the industry for ThoughtWorkers, and others who attend.
• In order to keep learning from what we do, and keep doing better in the future, we take feedback from attendees and speakers. Also, with an immediate after-the-event retrospective, we attempt to do the best for all participating in next vodQA.
• vodQA is a community event. Each community hosting vodQA should have the flexibility to structure the event in the way they see best suited for the attendees.
• Each vodQA should have a specific theme. This allows proper expectation setting from attendees, and also invites specific themed-topics from internal (ThoughtWorkers) and external speakers.

How we connect with like-minded people?

We started by using the database of people who had expressed interest in working with / being associated with ThoughtWorks, and spreading the event by word-of-mouth.

Then we started using the power of facebook (vodQA group), LinkedIn (vodQA group) and Twitter (#vodqa) to also spread the word, and bring the community together during non-event times, to share and collaborate with each other.

The journey so far

The first vodQA event in June 2010 at ThoughtWorks, Pune was a huge success. As a result, we immediately planned and hosted another vodQA in Oct 2010, again at ThoughtWorks, Pune.

The feedback and retrospectives provided us great motivation and inputs. As a result, we decided to make vodQA an India level conference.

As we took vodQA to other ThoughtWorks India offices out of Pune, (Bangalore, Chennai, Gurgaon), the respective community planned and executed this event, the way they felt best.

Till date, we have hosted the following vodQA events:

Theme	ThoughtWorks Location	Date
vodQA – The Testing Spirit	Pune	Jun-10
vodQA	Pune	Oct-10
vodQA	Pune	Mar-11
vodQA - Agile testing  for Enterprises	Gurgaon	Dec-11
vodQA - Continuous Testing For Total Quality Assurance	Chennai	Jan-12
vodQA - Agile Testing for Team and and Enterprises	Bangalore	Feb-12
vodQA – Testing and Beyond	Pune	Mar-12
vodQA – NCR	Gurgaon	Jun-12
vodQA Geek Night – Behavior Driven Testing (BDT) workshop	Pune	Jul-12
vodQA - The ABCs of testing - Automation, Big Data Analytics , Cloud Testing	Bangalore	Sep-12
vodQA – Going Beyond the Usual	Pune	Oct-12
vodQA Geek Night – Test Automation workshop	Pune	Apr-13
vodQA – Get, Set, Test	Bangalore	May-13
vodQA - Served on Mobile	Chennai	Jul-13
vodQA - Agility In Mobility	Gurgaon	Jul-13
*vodQA – Smarter | Faster | Reliable	Pune	Oct-13
*vodQA Gurgaon - Selenium for Beginners -	Gurgaon	Oct-13

*Upcoming events

Some interesting quotes from attendees:

• Liked the participation, interactive approach of the organizers and the passion and energy to make sure that audience goes satisfied
• A Saturday well spent @ThoughtWorks #vodQA. Got a chance to meet some highly motivated & inspired ThoughtWorkers & industry professionals!
• This is our QA family, and we want to see it grow

Overall we have had 800+ external (non-ThoughtWorker) participants and 75+ talks / workshops / games / sessions across all vodQA events. There have been a decent % of Developers / Business Analysts / Software Coaches attending and speaking, though the majority of attendees has been Software Testers.

I am very happy to see vodQA become popular and valuable to all passionate about Software Testing, all across India.

My dream is to take vodQA across the world as a community driven, Testing practioners conference!

Test Execution Trend feature added in TTA (0.5.7)

A new version of Test Trend Analyzer (TTA) is now available on github.

These are the new features in it:

• Test Execution Trend - to see the benchmarking of specific test execution over a time period

• Failure Analysis View

• Modified view to show failures for specific Test category along with specific test run

• Modified view to show failures for All test categories with latest test run

• Minor Refactoring for Compare Runs view

• Query optimisations + minor defect fixes.

For those that missed the earlier announcement - summary of features of TTA is available here, and source code is available on github.

PS: There is a lot of work (features / UI) planned for TTA. If you wish to contribute, we would love your help!!

vodQA - Faster | Smarter | Reliable - on Saturday, 19th Oct in Pune

After what seems to be ages, I am very happy to announce the next edition of  
vodQA - Faster | Smarter | Reliable coming to you on Saturday, 19th Oct, 2013 at ThoughtWorks, Pune.

In this edition of vodQA, the focus will be on cutting edge testing technology, which will help us to understand how to be more faster,smarter and reliable in our testing approach. It will ameliorate our testing strategy in a faster and smarter world , following the best practices at the same time. The emphasis will be more on discussing ideas which enable us to test smartly instead of testing more.

We are hoping to target following questions in coming vodQA edition:

- What all things I should start doing to be more effective and impactful Quality Analyst ?

- What all process, tools and techniques I can use to get more faster and yet reliable results ?

- Can I use the tool I am already using in some different way which will make my testing effort more effective, more faster and more reliable ?

- What can I do so that software I am working on is always in production readiness state ?

- Are there any ways in which I can reduce my test life cycle ?

For registration details, click here (http://bit.ly/18LSnjc).

For all past vodQA information, see here.

TTA (0.5.4) feature information and screenshots

Here are some screenshots from the current functionality supported by TTA v0.5.4.

See the TTA Wiki page or Contact me for more information on TTA.

Manual upload of Test data

Test Pyramid

Comparative Analysis

Compare different test runs

Test Failure Analysis

Integrate External Dashboards with TTA

TTA Statistics page

TTA - closer to becoming unified dashboard for Test Trends and Status

Test Trend Analyzer (TTA), went live again today !!! We are now at version 0.5.4.

In the past week we have made a bunch of improvements and added new features - Integrating External Dashboard , which takes TTA closer towards being the central dashboard for testing status across the organization.

 

There are a couple of projects within ThoughtWorks using this, and also some other organizations.

Here is the list of new Features:

• Integration of external dashboards (add from /admin page, see integration on /home page) - this allows one to integrate different existing dashboards into TTA - to make it a one stop place for seeing all Testing related information. Example: You can integrate your defect reports from Mingle / Jira / etc., or, you can also integrate your specific Continuous Integration (CI) dashboard from Go / Jenkins / Hudson / Bamboo / etc.
  
• Compare test runs (/compare_runs) - to compare specific test runs --- 
• what are the common failures, 
• what are the unique failures, 
• what failed on date 1, but passed on date 2
• what failed on date 2, but passed on date 1
• TTA Statistics Page (/stats) - to know usage of TTA by different projects / teams in your organization
• Fixes + minor UI modification

Features available from some time:

• Test Pyramid view (/pyramid) - to see how your project's automation effort aligns with the Test Automation philosophy
• Comparative Analysis view (comparative_analysis) - to see the trend of your test automation results over a period of time, and if any patterns emerge
• Failure Analysis view (/defect_analysis) - to make better meaning of the test failures, and help you prioritize which failures should be fixed first.
• Upload Test Run Data manually (/upload) - to manually upload test data in case if you have not uploaded test data automatically to TTA, but still want to use TTA

 

More information about features, how to use TTA, etc. can be found on this blog, TTA-github, TTA-github-wiki, or by contacting me.

About Me pages create, which do you like?

I have created 2 versions of About Me pages. Requesting your comments on which do you find better and more useful:

http://about.me/anand.bagmar

or,

http://essenceoftesting.blogspot.in/p/blog-page.html

TaaS @Agile2013

[UPDATE] Final update: The slides AND the audio recording (mp3) of my talk in Agile2013 are now available.

[UPDATE] Slides for the presentation are now available here.
 
So close, yet so far .... dreading the 15.30 hour non-stop flight from Mumbai to Newark, yet super excited to speak in Agile 2013, Integration testing in Enterprises using TaaS - via Case Study, on 5th August, from 2pm-3.15pm. Complete schedule is available here.

TaaS and BDT - back to back at WCNGT, Bangalore

I had a great time talking about TaaS and BDT at WCNGT conference in Bangalore on 11th July 2013.

Here is more information about the talks:

Integrated Testing in Enterprises using TaaS:

http://essenceoftesting.blogspot.in/2013/07/taas-in-wcngt-bangalore-and-agile-2013.html
The audio recording of the talk is available here.
The slides for the talk are available here.
TaaS is available on github and as a gem on rubygems.org.

Building the "right" regression suite using Behavior Driven Testing (BDT)

http://essenceoftesting.blogspot.com/2013/07/building-right-regression-suite-using.html
The audio recording of the talk is available here.
The slides for the talk are available here.

Building the "right" regression suite using Behavior Driven Testing (BDT)

As a last minute surprise, I have been requested to talk about Building the "right" regression suite using Behavior Driven Testing (BDT) in the World Conference on Next Generation Testing on 11/12 July 2013 in Bangalore. This is another of those topics I feel very strongly about and am very excited at this opportunity.

What this does mean though is that I have back-2-back 2 1-hour sessions at WCNGT2013 - first - Integrated Testing in Enterprises using TaaS, and immediately post lunch - the BDT talk. 

Should be fun ... and hopefully I don't mix up the content in the talks :)

Here is the high-level agenda of the topic: Building the "right" regression suite using Behavior Driven Testing (BDT):

• Overview of Agile Testing
• The Test Pyramid
• Different flavors of TDD
• BDD – Behavior Driven Development?
• ATDD – Acceptance Driven Development?
• BDT – Behavior Driven Testing?
• Difference between BDD and BDT
• Tools that support BDT
• The value proposition for the team - live documentation, business rules, onboarding, etc.

TaaS in WCNGT, Bangalore and Agile 2013, Nashville, USA

A busy couple of weeks for me as I talk about TaaS in World Conference on Next Generation Testing in Bangalore, on 11/12th July 2013, and in Agile 2013 in Nashville, USA, on 4-9th August 2013.
 
My session details are as follows:

In WCNGT - Integrated Testing In Enterprises Using TaaS, on 11th July, from 12pm-1pm. Detail program is available here.

In Agile 2013, Integration testing in Enterprises using TaaS - via Case Study, on 5th August, from 2pm-3.15pm. Complete schedule is available here.

vodQA mobile in July

July will see a strong dose of vodQA - focusing on Mobile Testing.

We start off with vodQA Gurgaon - Agility in Mobility on Thursday, July 11 2013 – 5.00 p.m. to 7.30 p.m.

Then, we have vodQA Chennai - Served on Mobile on Saturday, July 20 2013 – 09:30 AM - 4:30 PM.

Help share the word, and see you there!

A very intense BDT workshop in vodQA Bangalore

I, with the help of my colleagues, ran a BDT workshop in vodQA - Get, Set,Test! at Bangalore on 18th May 2013.

It was a very intense workshop, and challenging for us to execute as well for various reasons:

1. When we had run this workshop as part of vodQA Geek Night in July 2012, we had the opportunity to have a couple of dry runs - which allowed us to fine tune the content + pace ourselves better
2. The vodQA Geek Night session was 2.5-3 hours in duration
3. This time around, for vodQA Bangalore, all volunteers driving the workshop were in different locations. After the workshop was selected for this vodQA, we all met face-2-face directly on  the day of the event. So there was no question of a dry run before the event
4. The duration of the workshop this time around was 1.5 hours.

Despite all the challenges, I think we managed to deliver a good, solid content workshop and all the participants seemed very engaged, and there were a lot of good discussions.

Here are all the photos from vodQA - Get, Set, Test!

The slides used in the workshop are available here. Videos will follow soon.

For more info on past and upcoming vodQAs, please join our vodQA group on facebook.

I will be speaking about TaaS in Agile 2013 in Nashville, TN, USA

Very happy and excited to say that my talk on "Integration testing in Enterprises using TaaS - via Case Study" has been selected in Agile 2013. This conference is scheduled from 5-9th August in Nashville, TN, USA, and I get to present it as the 2nd talk on the 1st day of the conference!

Dartboard - Are you on track?

In Agile Projects, we use the swim-lanes to track the status of the card life-cycle in an iteration. Unfortunately, the swim-lanes depict a sequential work-flow. Something has to come first, second, ... last. That puts the thought in a lot of minds that what is first is indeed first, and what is last, well, is last in the scheme of things.

That depicts testing being done towards the end - which is very anti-agile!

 

Testing starts off way before development is completed on any card. See the "Agile QA Process" for one way to do Testing on Agile projects.

On Agile projects, r

eality is that testing is going on in some fashion or the other right from the beginning. To help bring that visibility into the work-stream, I tried creating a grid (physically on a board, also in mingle) - with the rows representing the state of testing in each swim lane ... but that too was not as appealing as the image shown below. I chose to call it the Dartboard.

Dartboard - Are you on track?

Some explanation on how we used this:

Each triangle can represent your individual swim lanes. From testing perspective, we chose to club together "ready for Dev", "in Dev" and "in UI" in the same category.

There is a specific in Testing triangle - because there is some amount of work that definitely needs to be done from testing perspective AFTER development and BA signoff is complete.

The RED triangle means the card is blocked from ALL perspectives in the iteration.

The GREEN tringle means the card is completed from ALL perspectives (analysis, dev, testing - manual + automation)

In each of the triangle (except RED and GREEN), the different colors mean specific things:

- BLACK band = Testing not yet started

- RED band = Testing is blocked (maybe for additional info needed, etc)

- BLUE band = Testing is in progress. Could be identifying test cases, doing manual / exploratory testing, setting up test data, automation, etc.

As the card moves between triangles, the testing state of each of the card is very visible. 

Add to this a simple time line on top / bottom of the card to indicate where you are in the iteration, and you know if your "state-of-panic" is justified or not at a very quick glance at the dartboard.

Some swim lane states we have used:

> Backlog / Spillover?

> Ready for Dev / In Dev / In UI

> Ready for BA Signoff / In BA Signoff

> Ready for Test

> In Test / In Integrated Test

> Done

> Blocked

We also tracked each type of card separately:

Defects = RED cards / stickies

Story cards = Blue / Yellow cards / stickies

Tech cards = White cards / stickies

TaaS is available on rubygems.org

I have finally created and released the TaaS gem to rubygems.org. You can find it here.

To know more about TaaS - see all related blog posts here, or refer to the github project for the same.