Tuesday, July 19, 2016

Slides & Video from Test Data - Food for Test Automation Framework

As posted earlier, I spoke about "Test Data - Food for your Test Automation Framework!" at Selenium Conference 2016 in Bangalore on 24th June 2016 in front of a packed and a very interactive audience. What a great time it was!

The video for that session is finally available! I have updated the original post as well as linked the slides & video with this post as well.

Video



Slides



Wednesday, July 6, 2016

Any browsermob-proxy users facing issues with some requests not getting fired?

Is there anyone using browsermob-proxy who is having issues with some requests not getting fired?

I have integrated browsermob-proxy with my protractor tests. This works wonderfully when I run my tests from Mac (against local environment, or any other test environment).

However, when I run my tests from CI (agent is SUSE Enterprise 11.4) - my tests fail. 

I narrowed down the problem to the following scenario:

On some specific user action in the UI, there are a lot (>100) of requests fired from the browser in parallel (batches). There are a couple of scenarios like this in my application - and the test fails in all these cases.

Here is a screenshot of what the captured HAR file shows -



The same test works when I run this locally from Mac

Any idea how to fix this? Thank you in advance!

See this issue for more details - (https://github.com/lightbody/browsermob-proxy/issues/492)


Tuesday, July 5, 2016

Any WAAT (Web Analytics Automation Testing Framework) users out there?

It has been over 2 years since any update to WAAT - Java or Ruby. Over the years, I have realised, and also received a lot of thoughts / feedback from users of WAAT around where it helps, and what challenges exist. 

Also, given the widespread IoT & Big Data based work going on around the world, (Web) Analytics now plays a much bigger role in guiding business take better decisions. 

WAAT (again) fits in the grand scheme of things very nicely as a framework to automate the validation of correct reporting of tags to any Web Analytics solution provider.

Hence, its a no-brainer for me - it is high time I work on some of the feedback and limitations of WAAT to make it usable again!

At the recently concluded Selenium Conference 2016 held in Bangalore, India, I got an idea of how to overcome a lot of challenges (listed below) and pain in using WAAT. 


What's next?

To implement my new idea, this does mean a couple of things:

  • Existing plugins have limited use - and needs to be deleted.
  • A new plugin would need to be created - which may mean different set of APIs, and also different way to specify the test data.

Questions for you

Before I go ahead making these changes - I would like to get answers to the below questions (please add your answers directly in the comments):
  • Is anyone currently using WAAT? If yes - 
    • which version (Java / Ruby)?
    • which plugin
    • Using HTTP / HTTPS?
    • Which Web Analytic solution are you using? (ex: Google Analytics, WebTrends, etc?)
  • Would you be interested in using the new WAAT? If yes - 
    • Which language? Java / Ruby / JavaScript / Python / etc?
  • Would you like to contribute to implementing this new WAAT? If yes - contact me! :)
-----------------------------------------------------------------------------------------------------------------

Current plugins available in WAAT:

  • Omniture Debugger (WAAT-Java)
    • Pros:
      • OS independent
      • Run using regular-test-user 
    • Cons:
      • Browser dependent - need to implement ScriptRunner for the UI-driver in use
      • Web-Analytic solution dependent - only for Adobe Marketing Clout / Omniture SiteCatalyst
  • HTTPSniffer (WAAT-Java, WAAT-Ruby)
    • Pros
      • Web-Analytic solution independent
      • Browser independent
      • UI-Driver independent
    • Cons
      • 3rd party libraries are OS dependent
      • HTTPs is not supported out-of-the-box
      • Run tests as "root"
  • JSSniffer (WAAT-Java, WAAT-Ruby)
    • Pros
      • Web-Analytic solution independent
      • Browser independent
      • HTTPs supported out-of-the-box
      • No 3rd party library dependency
    • Cons
      • Need to write JavaScript to get the URL from the browser context
      • UI-Driver dependent
-----------------------------------------------------------------------------------------------------------------

Saturday, July 2, 2016

What is Web Analytics and how to Test it?

vodQA returned - this time with the theme - Testing Heuristics in ThoughtWorks, Hyderabad on 2nd July 2016.

Here I spoke about - "The What, Why and How of Web Analytics Testing". 

Abstract

Learn what is Web Analytics, why is it important, and see some techniques how you can test it manually and and also automate that validation. But just knowing about Web Analytics is not sufficient for business now.
There are new kids in town - IoT and Big Data - two of the most used and heard-off buzz words in the Software Industry!

With IoT, with a creative mindset looking for opportunities and ways to add value, the possibilities are infinite. With each such opportunity, there is a huge volume of data being generated - which if analyzed and used correctly, can feed into creating more opportunities and increased value propositions.

There are 2 types of analysis that one needs to think about.
1. How is the end-user interacting with the product? This will give some level of understanding into how to re-position and focus on the true value add features for the product.
2. With the huge volume of data being generated by the end-user interactions, and the data being captured by all devices in the food-chain of the offering, it is important to identify patterns from what has happened, and find out new product / value opportunities based on usage patterns.

Slides


Video

PS: Apologies for the video quality - I am not seen very clearly - but the slides are bright & clear, and so is the audio - so the important aspects are covered!




Pictures









Feedback


My Takeaway & Learning

- The attendees did not have much exposure to Web Analytics, and how it works. I should spend more time in speaking about that
- I should spend more time in challenges and potential solutions related to Big Data & IoT
- A lot of people are interested in WAAT - that could be a separate, more detailed discussion

Thursday, June 30, 2016

Learnings from Selenium Conference 2016, Bangalore

The value one gets by attending any conference / training / meetup / etc. is subjective to various aspects, some of which are mentioned below (in no particular order):

  • Individual skills & capabilities
  • Past experiences
  • Existing knowledge / information / expertise on the subject 
  • Open mindedness
  • Willingness to learn
  • Current work (tools & tech stack, challenges, risks, priorities, backlog, tech debt, team members, etc.)

The above aspects definitely played a part in what takeaways I had from the recently concluded Selenium Conference 2016 in Bangalore as well.

Here are my key takeaways, which I am going to work on learning more about, or implementing in the near future - special thanks to +Dave Haeffner , +Marcus Merrell , +Simon Stewart+Bret Pettichord for helping me find these takeaways as part of various conversations during these few days.


  • Related to Protractor
    • Use Proxy Server in tests (Protractor framework) to capture HAR file on specific actions (AJAX calls) - and capture performance metrics from the same
    • Read and experiment with the Marionette driver for Firefox - maybe it helps me overcome some of my challenges with Firefox & Maps in CI environment (headless using xvfb)
    • Remove "phantomJS" as a supported browser from my framework by ensuring headless tests work with Chrome & Firefox using xvfb
    • Highlight element when running tests before taking screenshots - will help in debugging
    • Experiment with different loggers & reporters - Allure, Winston logger
    • Better "promise" handling in framework to keep abstraction layers sane
  • Revive WAAT - Web Analytics Automation Testing Framework - create new plugin using Proxy Server approach. Also remove Omniture Debugger and HttpSniffer plugin.
  • Refocus energy on TTA - Test Trend Analyzer.
  • Keep vodQA going strong - its a good community initiative

See you all in Selenium Conference UK in November 2016!


Tuesday, June 28, 2016

Taking Protractor to the next level

As I got onto a project early this year - my paths crossed again with Protractor / Angular / JavaScript. Despite my past experiences with this tech stack, I needed to remain positive when approaching this challenge.

In this case, the context was quite different. That said, the challenges, though reduced compared to my first attempt at using Protractor effectively, were still around. 

I had posted a set of questions on my blog, various LinkedIn and facebook groups - and got a lot of suggestions and advice how to proceed. Thanks to all of them, it did make my life easier.

Then, as part of Selenium Conference 2016 held in Bangalore on 24-25th June 2016, my friend and colleague - +Nikita Iyer and I spoke about "Sharing the Pain of Automating with Protractor & WebDriver". The intention was multi-fold:
  • We wanted to share what challenges we had faced on the team when using Protractor, and also how we overcame certain challenges. This was to help others in similar situations to learn from our mistakes & solutions.
  • We also wanted to learn from the attendees what other challenges & solutions they had come up with in their experiences with this Protractor tool-stack.
I am very happy to say that we were successful in achieving both these objectives quite successfully.

Below is the details from the talk.

Abstract

There is a saying ..."Sukh baatne se badhta hai, dukh baatne se kam hota hai", translated as - "happiness increases & sadness reduces on sharing with others".

We want to take this opportunity to share with our experiences - the good and the bad, in the journey of building a Test Automation framework for an AngularJS based application. 

We will learn, by a case study, what thought process we applied on the given context (product, team, skills, capabilities, long term vision) to come up with an appropriate Test Automation Strategy. This Test Automation strategy covered all aspects of Test Automation - Unit, Integration, UI - i.e. End-2-End tests (E2E).

Next, we will share how we went about narrowing-down, and eventually selecting a specific Tech Stack + Tools (Javascript / Jasmine / Protractor / Selenium-WebDriver) to accomplish the Test Automation for the product.

Lastly, we will share the challenges that came up in the implementation of the Test Automation, and how we overcame them. This will also include how we managed to get the tests running in Jenkins - a Continuous Integration tool. 
This discussion is applicable to all team members who are working on Test Automation!

P.S. We will be attempting to make a sample protractor-based automation framework available on github for anyone to use as a starting point for setting up protractor-based Test Automation framework.


Slides


Video

- will be updated once available

Pictures




Monday, June 27, 2016

The recurring pattern of Patterns

On popular demand, I spoke in TechJam, the 2nd time around on Tuesday, 21st June on "Patterns of a 'good' Test Automation Framework, Locators & Data".




This time, the TechJam team at IDeaS used a different setup - a new conference room, with live-streaming via Google Hangout. It was indeed a great setup - with around 40 people able to sit and participate in discussions for over an hour - about Patterns in Test Automation.




Slides


Video


Pictures




Saturday, June 25, 2016

Test Data - Food for Test Automation Framework at Selenium Conference 2016

[Updated - Pictures added, Video added]

I spoke about "Test Data - Food for your Test Automation Framework!" at Selenium Conference 2016 in Bangalore on 24th July 2016 in front of a packed and a very interactive audience. What a great time it was!

Here is some information about the session - 

Abstract

Building a Test Automation Framework is easy - there are so many resources / guides / blogs / etc. available to help you get started and help solve the issues you get along the journey.

Teams already building 1000s of tests of various types - UI, web service-based, integration, unit, etc. is a proof of that.

However, building a "good" Test Automation Framework is not very easy. There are a lot of principles and practices you need to use, in the right context, with a good set of skills required to make the Test Automation Framework maintainable, scalable and reusable.
In this talk, we will focus on one of the critical aspects and patterns in building the Test Automation framework - Test Data!

We will look at different data patterns as options and techniques how to create, manage, use, reuse Test Data in a way to keep the tests running in an reliable and deterministic way. We will also discuss what questions to ask, what things to think about in selecting your approach for Test Data!

This discussion will be applicable for any type of Test Automation (web / mobile / desktop), but, we will focus primarily on UI automation frameworks, ex. using Selenium.

Slides

Video


Pictures



Monday, April 25, 2016

Patterns of Test Automation in ATAGTR2016

ded>
 
I spoke about "Patterns in Test Automation Framework" in ATA's Global Testing Retreat 2016, held in Pune. For a change, I did not need to travel out of Pune to speak in a conference :)

Here are some details about the same:


Abstract

Building a Test Automation Framework is easy - there are so many resources / guides / blogs / etc. available to help you get started and help solve the issues you get along the journey. However, building a "good" Test Automation Framework is not very easy. There are a lot of principles and practices you need to use, in the right context, with a good set of skills required to make the Test Automation Framework maintainable, scalable and reusable. Design Patterns play a big role in helping achieve this goal of building a good and robust framework.
In this talk, we will talk about, and see examples of various types of patterns you can use for:1. Build your Test Automation Framework2. Test Data Management3. Locators / IDs (for finding / interacting with elements in the browser / app)Using these patterns you will be able to build a good framework, that will help keep your tests running fast, and reliably in your CI / CD setup!


Slides




Video


Feedback from attendees

Here is some feedback I received from the attendees:


Few remarks that we received from the attendees:
1.          Learnt business driven page objects
2.       Thoughtful presentation. Sound knowledge
3.       Very clear and concise. Meaningful framework

Friday, March 18, 2016

Patterns in Test Automation presented at Agile India 2016

I spoke about Patterns of a “good” Test Automation Framework, Locators & Data! in Agile India 2016 in front of a packed room of Developers, Testers, BAs and POs.

Below is the abstract, slides & video of the talk.


Patterns of a “good” Test Automation Framework, Locators & Data!

Building a Test Automation Framework is easy - there are so many resources / guides / blogs / etc. available to help you get started and help solve the issues you get along the journey.
However, building a "good" Test Automation Framework is not very easy. There are a lot of principles and practices you need to use, in the right context, with a good set of skills required to make the Test Automation Framework maintainable, scalable and reusable.
Design Patterns play a big role in helping achieve this goal of building a good and robust framework. 
In this talk, we will talk about, and see examples of various types of patterns you can use for:
  1. Build your Test Automation Framework
  2. Test Data Management
  3. Locators / IDs (for finding / interacting with elements in the browser / app)

Learning Outcome

  • Patterns for building Test Automation Framework
  • Patterns for Test Data Management, with pros and cons of each
  • Patterns for managing locators / IDs for interaction with UI


Slides

Slides are available here:

Video

(My attempt of capturing the) video is available here:


Official video will be linked when it gets available.