Monday, May 19, 2014

WAAT at StarEast2014

I was speaking about "Build the 'right' regression suite using Behavior Driven Testing (BDT)" at StarEast 2014 and met Marcus Merrell who was speaking about "Automated Analytics Testing with Open Source Tools". I figured out deep into our conversation that he has used WAAT, and a few others at the table also were aware of and had used WAAT before. Felt great!

Thursday, May 15, 2014

Update from StarEast 2014 - "Build the 'right' regression suite using BDT"

I had a great time speaking about how to "Build the 'right' regression suite using Behavior Driven Testing (BDT)".

This time, I truly understood the value of practice, dry-runs. Before the webinar for NY Selenium Meetup group, I presented the content to my colleagues in the ThoughtWorks, Pune office and also shared the same with the hivemind @ ThoughtWorks and got great feedback. That helped me tremendously in making my content more rock-solid and well-tuned.

As a result, I was able to deliver the content, to a very enthusiastic and curious audience, who turned up in great numbers (>105) to hear about what the $@$#^$% is BDT, and how can it help in avoiding the nightmare of long, unfruitful, painful Regression Test cycles.

The slides from the talk are available here and the video is available here.

Thursday, May 8, 2014

Update from Webinar on "Build the 'right' regression suite using BDT" for NY Selenium Meetup

I had a challenging, yet good time speaking in a Webinar for the New York Selenium Meetup community on how to "Build the 'right' regression suite using Behavior Driven Testing (BDT)". This webinar was conducted on 6th May 2014 at 6.30pm and I am very thankful to Mona Soni to help organize the same.

Before I speak about the challenges, here are the slides and the audio + screen recording from the webinar. The video is not cleaned-up ... I had started recording the session and then we did wait for a few minutes before we started off, but you can forward to around the 01:15 min mark and audio starts from that point.

This was challenging because of 2 main reasons:
> With a webinar, I find it difficult to connect with the audience. I am not able to gauge if the content is something they already know about, so I can proceed faster. Or, if they are not following, I need to go slower. Or, the topic is just not interesting enough to them. There may be other reasons as well, but I just do not get that real-time feedback which is so important when explaining a concept and a technique.
Though there were some good interactions and great questions in form of chat, I miss that eye-to-eye connect. This webinar was conducted using GoToMeeting. Maybe next time I do this, I need to try to get webcams enabled for atleast a good few people attending to understand that body language.

> The 2nd challenge I had was purely my own body not being able to adjust well enough. I had flown in from India to Florida to speak in STAREAST 2014 conference just a couple of days ago, and was still adjusting to the jet-lag. Evening times turned out to be my lowest-energy points on the day and I felt myself struggling to keep focus, talk and respond effectively. I would like to apologize to the attendees if they felt my content delivery was not up to the mark for this reason.

I appreciate any feedback on the session, and looking forward to connect with you and talk about Testing, Test Automation, my open-source tools (TaaS, WAAT, TTA) and of course BDT!

Monday, April 21, 2014

What are the criteria for determining success for Test Automation?

Test Automation is often thought of as a silver bullet that will solve the Teams testing problems. As a result there is a heavy investment of time, money, people in building an Automated Suite of Tests of different types which will solve all problems.

Is that really the case? We know the theoretical aspect of what is required to make Test Automation successful for the team. I want to know from practical perspective, with the context what worked or not for you.

I am currently writing about "Building an Enterprise-class Test Automation Framework", and am gathering experience reports based on what others in the industry have seen.
 
I am looking for people to share stories from their current / past experiences of full or limited success of Test Automation, answering the below questions (at a minimum):
 

Context:

  • What is the product under test like? (small / med / large / enterprise) (web / desktop / mobile / etc.)
  • How long is the the Test Automation framework envisioned to be used? (few months, a year or two, more than a few years, etc.)
  • What is the team (complete and test automation) size?
  • Is the testing team co-located or distributed? 
  • What are the tools / technologies used for testing?
  • Are the skills and capabilities uniform for the team members?
  • Is domain a factor for determining success criteria?  

Framework related:

  • What are the factors determining the success / failure of Test Automation implementation?
  • What worked for you? 
  • What did not work as well?
  • What could have been different in the above to make Test Automation a success story?
  • What are the enablers in your opinion to make Test Automation successful?
  • What are the blockers / anchors in your opinion that prevented Test Automation from being successful?
  • Does it matter if the team is working in Waterfall methodology or Agile methodology?
Anything else you can think of?

Thursday, April 10, 2014

Sample test automation framework using cucumber-jvm

I wanted to learn and experiment with cucumber-jvm. My approach was to think of a real **complex scenario that needs to be automated and then build a cucumber-jvm based framework to achieve the following goals:
  • Learn how cucumber-jvm works
  • Create a bare-bone framework with all basic requirements that can be reused
Once you know the basics and fundamentals of building a scalable and maintainable Test Automation frameworks, it was really easy to apply my past learning and experiences to learn cucumber-jvm and build a framework from scratch.

So, without further ado, I introduce to you the cucumber-jvm-sample Test Automation Framework, hosted on github. 

Following functionality is implemented in this framework:

  • Tests specified using cucumber-jvm
  • Build tool: Gradle
  • Programming language: Groovy (for Gradle) and Java
  • Test Data Management: Samples to use data-specified in feature files, AND use data from separate json files
  • Browser automation: Using WebDriver for browser interaction
  • Web Service automation: Using cxf library to generate client code from web service WSDL files, and invoke methods on the same
  • Take screenshots on demand and save on disk
  • Integrated cucumber-reports to get 'pretty' and 'meaningful' reports from test execution
  • Using apache logger for storing test logs in files (and also report to console)
  • Using aspectJ to do byte code injection to automatically log test trace to file. Also creating a separate benchmarks file to track time taken by each method. This information can be mapped separately in other tools like Excel to identify patterns of test execution.

Feel free to fork and use this framework on your projects. If there are any other features you think are important to have in a Test Automation Framework, let me know. Even better would be to submit pull requests with those changes, which I will take a look at and accept if it makes sense.

** Pun intended :) The complex test I am talking about is a simple search using google search.

Tuesday, April 8, 2014

Friday, March 28, 2014

WAAT Java v1.5.1 released today

After a long time, and with lot of push from collaborators and users of WAAT, I have finally updated WAAT (Java) and made a new release today. 

You can get this new version - v1.5.1 directly from the project's dist directory.

Once I get some feedback, I will also update WAAT-ruby with these changes.

Here is the list of changes in WAAT_v1.5.1:

Changes in v1.5.1

  • Engine.isExpectedTagPresentInActualTagList in engine class is made public
  • Updated Engine to work without creating testData.xml file, and directly sending exceptedSectionList for tags
    Added a new method
        Engine.verifyWebAnalyticsData(String actionName, ArrayList
    expectedSectionList, String[] urlPatterns, int minimumNumberOfPackets)
  • Added an empty constructor for Section.java to prevent marshalling error
  • Support Fragmented Packets
  • Updated Engine to support Pattern comparison, instead of String contains
Do let me know if you see any problems / issues with this update.

Thanks.

Thursday, February 27, 2014

Automate across Platform, OS, Technologies with TaaS

[Updated - link to slides, audio, experience report added]

The talk at Agile India 2014 went really well. A few things happened:
  • My talk was the 2nd last talk on a Saturday. My hopes of having a decent sized audience was low. But I was very pleasantly surprised to see the room almost full.
  • Usually in conferences I have spoken at, the ratio of technical / hands-on people Vs leads / managers is around 20:80. In this case, that ratio was almost inverted. There were about 70-80% technical / hands-on people in the audience.
  • Due to the higher technical audience, there were great questions all along the way - which resulted in me not able to complete on time ... I actually went over by 5-6 minutes and that too had to really rush through the last few sections of my presentation, and was not able to do a complete demo.
  • Almost everyone was able to relate to the challenges of integration test automation, the concept and the problem TaaS solves - which was a great validation for me!
  • Unfortunately I had to rush to the airport immediately after the talk, which prevented me from networking and talking more specifics with folks after the talk. Hopefully some of them will be contacting me to know more about TaaS!

The slides from the talk are available here on slideshare. You can download the audio recording from the talk from here. The pictures and video should be available from Agile India 2014 site soon. I will update the links for the same when that becomes available.

--------

After what seems to be a long time, I am speaking in Agile India 2014 in Bangalore on  "Automate across Platform, OS, Technologies with TaaS".

I am changing the format of the talk this time and am hoping to do a good demo instead of just showing code snippets. Hopefully no unpleasant surprises crop up in the demo!! 

Other than that, really looking forward to interacting with a lot of fellow-enthusiasts at the conference.

Slides and experience report to follow soon.