Converting JSON into usable objects

JSON is a great way to specify data / information and, off late, it is the format of my choice to specify test data.


I find it to be -

• light weight 
• easy to understand 
• almost very intuitive to know if you have made an error in the syntax 
• easy to read into code and parse 
• easy create meaningful custom objects and use in code 

Recently, thanks to a friend - Abhijeet Vaikar, I came across a tool - quicktype.io - that helps in transforming the raw JSON (from various sources) directly into custom objects, in a variety of languages.

Site: https://quicktype.io/

The tool: https://app.quicktype.io/

I got to know about this tool at perfect time as I am building a new tool for dynamic logging in Java - AutoLogJ (but more about AutoLogJ later). quicktype.io does what it promises - and it saved me a lot of time to build the custom POJOs for the same.

Thanks Abhijeet Vaikar and the quicktype team!

A few thoughts on Test Automation

Deepanshu Agarwal and Brijesh Deb asked some very interesting questions on a LinkedIn post. Since I have some verbose thoughts on this, thought it is better to respond via a blog post instead.

• Why is Test Automation still considered suitable only for regression testing? What about writing automation tests sooner as in case of Test Driven Development?
• [Anand] - Depends what you call test automation? If ONLY FUNCTIONAL, then its better to explore the product first, investigate / have conversations with developers on what lower-level tests are already automated, and then based on cost / risk-value analysis, decide what else needs to be automated at Functional layer.
  
  
  A tangential rant ....

  The reason we think about classifications such as SMOKE, SANITY, REGRESSION in Functional Automation ONLY has a big reason. These tests are inherently very slow, brittle and it takes a lot of effort to ensure these tests give poor feedback on exact point / reason of failure. 

  
  

  I have never seen any other form of tests - say Unit tests, which would be magnitudes in number larger than the functional tests (hopefully) ever have any such classification. We all just say, the unit tests ran, not the smoke unit tests ran. 

  
  

  We need to grow up and understand the reason behind this. We need to make our top-of-the-pyramid tests as less in number as possible. We need to ensure we use good programming / development practices and get quick and reliable feedback from these tests. Else we will keep focussing on the symptoms, and never get to the root cause.

  
  

  --- Rant ends

  
  

  Once we understand this, then it is a matter of understanding in the context what can and needs to happen first, and what next. In most cases TDD will work. But TDD as a Functional Spec may, or may-not be an overkill .... the team has to decide that.

• Why do the automated tests always have to derived from manual tests?
• [Anand] - What is a manual test? Something that a machine is not performing? How do you do "manual testing"? Is Exploratory Testing subset of Manual Testing, or the other way, or any other thoughts on that?
  
  
  From the perspective of "automated tests" - I read it as "automated functional tests" here. In that case, the answer for the above question holds true here as well.
  
  
  Continuing from that thought - I think the approach (of deriving automated tests from so-called manual tests) is better than thinking upfront what tests I am going to automate and then proceed with the implementation without any thought or regard to any other learning along the way.

• The tests classified as manual tests are only focused at ensuring certain checks. What about actually running some tests to discover the unknown?
• [Anand] I don't want to get into the 'checks' debate. It is futile!
  
  
  All I have understood is - you cannot just spend time looking at the requirements / specs and write down (in your mind / bullet points / story cards / some fancy ALM tool) your test cases / scenarios. 
  
  
  That list is just a starting point of your journey of exploration and experimentation with the product-under-test. If you think that what you have identified is your actual scope of testing, then ALL THE BEST to you, your team and your product - because there are going to be so many opportunities you have missed out to make the product better and usable for the end-user. Unfortunately, lot of organisation still look for "regression" testing cycles - where (you think) you execute all the tests that were identified in a time long ago. However, everyone knows, it is best case / best effort, IF AT ALL, of actually following each and every step of that regression cycle. Such a waste of time and effort - when more meaningful testing could have been performed during that time.

• Why is that exploratory tests are still considered suitable only for manual testing? How about automating exploratory tests using AI?
• [Anand] What is the meaning of "exploration"?
  As per a quick online search, this is what it means:
  
  
  
  
  
  Now - how can you automate the unknown / unfamiliar? You can use tools to help figure out what is unknown / unfamiliar ... but once you know it, then it does not remain 'unknown'. I think buzzwords like AI and ML are tools to help bridge the gap in the known and the unknown. But we would still need to guide and use these tools and technologies to our advantage, to aid in our exploration.
  
  

Implementing Soft Assertions

Back in 2009 / 2010, I was working on implementing end-2-end tests for a web site using Java / Selenium / TestNG based automation. The challenge I was facing was that the tests used to fail for trivial (but valid) reasons - and I wondered that the test did not even get to core validations before it failed. How will the team ever know about the main issues in the product if the test fails for trivial issues? 

That was a trigger point for me to think about Soft Assertions - what if there was a way to say if there is a type of failure that I want to know about, but the test can continue with the remaining set of validations - unless something does not make sense to proceed with.

Ex: If the text message of a field is incorrect, I can continue. But if login fails, no point in proceeding with the rest of the test.

This idea seemed interesting - so I came up with the following requirements from such an  implementation as listed below:

• Clear distinction between what type of failure I can continue from, or not
• Ex: assert.** is for hard asserts. verify.** is for soft asserts
• All failures that I can continue from (i.e. soft asserts), need to be collated and at the end of the test, the complete list of those soft assert failures should be presented with the test result (and in the report), while the test failed just once
• Ex: There were 5 soft assertion failures)
• Capture relevant screenshots whenever Soft Assertion failed
• If there was a hard assert along the way of the test execution, the test failure should include the prior soft assert failures along with the hard assert failure, as appropriate

For the actual implementation, I did the following (in 2009/2010):

• I looked into the TestNG code base, and I could not really find any out-of-the-box support for what I wanted to do. 
• So for lack of knowledge on better ways of implementation, 
• I checked-out the TestNG code, 
• added the Soft Assertion implementation, and, 
• built a custom TestNG.jar file
• checked-in the jar file as a library artefact in our automation framework. 

In hindsight, I should have sent that functionality as a PR to the project. 

But not all is lost, TestNG now (or maybe since some time now) has support for Soft Assertions - out-of-the-box. And it is pretty straightforward to implement / use it as well.

Implementing Soft Assertions in your test framework

See this gist for implementation that you can you use with TestNG (I tested with v6.10).

Using Soft Assertions in your tests

Here is how you can use the Soft Assertions in your tests.

Soft Assertions in any other tech stack?

What if you are not using TestNG, or Java - rather, what if you are using completely different programming language / tools / test-runner? Can you still use Soft Assertions? 

Absolutely YES! All you need to understand is the concept, and figure out the best way to implement the same, if any out-of-the-box solution does not exist in that tech stack. 

Hope this helps you!

Return of the todo (learning) list

Since at least a decade, I have had a list of TODOs which I actively updated and maintained. The items on this list focused on - 

• what new things I wanted to learn / experiment with
• new conference talk ideas
• open source ideas / updates

Unfortunately, due to various reasons (some of which were my doing as well), the focus on learning and experimentation got lost ... and I felt awful about it. I also shared my pain with some friends and colleagues of unable to find time / opportunity / focus / support to be in the continuous learning phase. But not much could be done / changed in those circumstances.

But I am very happy to share that the list is back, and back with a bang!!!

The days of learning and experimentation continue. My list is now overflowing, and continuing to grow with ideas and things I want to learn and experiment.

I am happy again!!

Essence of Testing - A new beginning

In my career so far, I have been very fortunate to have got great mentors, and a variety of opportunities to learn, add value, and share my experiences with others around me.

Here are some of these experiences:

• Worked in various sized organisations across the globe in the past couple of decades
• The teams have been big and small
• Played a variety of roles - Quality Analyst, SDET (Software Development Engineer in Test), Product Quality Engineer (PQE), Automation Engineer, Consultant, Coach, Project Manager, Director - Quality, Support Engineer, etc.
• Worked with teams having products in different domains - Health care, eCommerce, Banking / Finance, Retail, Entertainment (OTT), Research, etc. 
• With in organisations (B2B and B2C space): WebMD, Borland Software, Microsoft (Redmond, USA), AmberPoint (USA / India), ThoughtWorks, Vuclip 
• Shared experiences with other via Meetups and Conferences world-wide
• Created opensource tools like WAAT, TTA, TaaS

After working as an employee in these organisations for almost a couple of decades now, I have taken a plunge to do something different.

I am now looking forward to work with Organisations and Teams, to help co-create optimized solutions towards shipping a quality product. Leave me a message on my blog, or send me an email at abagmar@gmail.com to talk more on how we can work together!

As part of this journey, even before I was able to buy my own laptop, and warm it up, I got an interesting consulting assignment, thanks to a dear friend.

This assignment was exactly what I needed to get started - it was a Discovery Workshop, with the following objective:

• Learn and understand the current state of the team and the product they were building
• Understand current (perceived) challenges
• Suggest improvements for the team in areas of:
• Tweaks in the current processes
• Practices to be adopted / got better at / or stopped (as anti-patterns and not adding value)
• Identify opportunities to Test better, and early
• Suggest a Test Automation Pyramid that is fit-for-purpose for the team
• Suggest strategy, tools and approach for end-2-end (e2e) functional automation

As I usually do, I started off the workshop with my favorite tool for note taking - Mind Maps! As the conversations evolved and got specific, I started with a Balanced Mind-Map  to Fishbone Mind-Map.

Eventually I consolidated my thoughts and created the Discovery Workshop Report for the client in the form of a slide deck, using the following approach.

• Learning from Discovery
• Talk with the Team members
• See & understand the project management tool, quality of requirements, etc.
• See the code - understand complexity, types of tests written, quality of tests, etc.
• See the Testing related artifacts - test cases, test execution strategy, exploratory testing, etc.
• See the CI server - how deployments happen, what causes the builds to fail, etc.
• Recommendations
• For all the above areas, I created recommendations on what different aspects may help the team move ahead in a better way

Discovery and Recommendations were based on each specific activity:

• Process
• Architecture
• Requirements
• Development
• Testing
• Deployments

Once the learning from the Learning and Recommendation stages were well understood (by me), I then created a Suggested Plan of Execution (in phases / milestones)

• From the recommendations, I created milestone based plans on what can be started immediately, and what decisions the team needs to take to move forward in other areas

In this 2 week time, unknowingly, in retrospect, the whole engagement was quite agile. There were periodic demos of the POCs, regular progress sharing, and changing direction of discovery and recommendations based on learning.

The Client also appreciated the quality of conversations, and the results that were shared.

All in all, a very satisfying beginning to my new journey with Essence of Testing!

Test Driven Development (TDD) and its modern variations

NOTE / RECOMMENDATION - This blog post is to be read with your funny side switched on!

First - lets answer the question - What is TDD?

Directly from https://en.wikipedia.org/wiki/Test-driven_development - Test-driven development (TDD) is a software development process that relies on the repetition of a very short development cycle: Requirements are turned into very specific test cases, then the software is improved to pass the new tests, only. This is opposed to software development that allows software to be added that is not proven to meet requirements.

American software engineer Kent Beck, who is credited with having developed or "rediscovered"^[1] the technique, stated in 2003 that TDD encourages simple designs and inspires confidence.

Test-driven development is related to the test-first programming concepts of extreme programming, begun in 1999,^[3] but more recently has created more general interest in its own right.

However, over the years, fortunately or unfortunately, I have also come across various different techniques (used consciously / unconsciously) in Software Development, to get the work done. Some of them are listed below - in alphabetical order:

BDD = Behavior / Business Driven Development
BDD = Blame Driven Development
BDD = Buzzword Driven Development
CDD = Calendar Driven Development
CDD = Checklist Driven Development
CDD = Chadi (stick) Driven Development
CDD = Constraint Driven Development
DDD = Date Driven Development
DDD = Defect Driven Development
DDD = Document (PRD) Driven Development
EDD = Escalation Driven Development
EDD = Estimation Driven Development
EDD = Excel Driven Development
FDD = Fashion Driven Development
FDD = Fear Driven Development
FDD = Footwear (punishment) Driven Development
HDD = Hope Driven Development (fingers crossed)
IDD = Instinct Driven Development
IDD = Issue Driven Development
JDD = Jira Driven Development
MDD = Metrics Driven Development
MDD = Manager / Management Driven Development
PDD = Patch(work) Driven Development
PDD = Plan Driven Development
PDD = Prayer Driven Development
PDD = Process Driven Development
RDD = Resource Driven Development
RDD = Resume Driven Development
SDD = Stackoverflow Driven Development
SDD = Stakeholder Driven Development

And the last one -
NDD = No-Drive (towards) Development

Any other style of development you have come across in your experience?

Hope you had fun reading the list! 

Measuring Consumer Quality - The Missing Feedback Loop

I spoke in vodQA at ThoughtWorks, Pune on "Measuring Consumer Quality - the Missing Feedback Loop". 

This talk address the why and how from my earlier blog post on "Understanding, Measuring and Building Consumer Quality". I recommend you read that first, before going through the slides and video for this talk.

Abstract:

How to build a good quality product is not a new topic. Proper usage of methodologies, processes, practices, collaboration techniques can yield amazing results for the team, the organisation, and for the end-users of your product.

While there is a lot of emphasis on the processes and practices side, one aspect that is still spoken about "loosely" - is the feedback loop from your end-users to making better decisions.

SO, What is this feedback loop? Is it a myth? How do you measure it? Is there a "magic" formula to understand this data received? How to you add value to your product using this data?

In this interactive session, we will use a case study of a B2C entertainment-domain product (having millions of consumers) as an example to understand and also answer the following questions:

• The importance of knowing your Consumers 
• How do you know your product is working well? 
• How do you know your Consumers are engaged with your product? 
• Can you draw inferences and patterns from the data to reach of point of being able to make predictions on Consumer behaviour, before making any code change? 

Video:

Slides can be found here.

Pictures:

MAD-LAB - Capabilities & Features - Agile India 2018

I spoke about "Build your own MAD-LAB - for Mobile Test Automation for CD" at Agile India 2018.

Though I have spoken on this similar topic answering the question - "Why I needed to build my own MAD-LAB?" before at vodQA in July 2017 at Vuclip, quite a few things have changed since then.

Knowing the value of "being agile", a day before my scheduled talk in Agile India 2018, I decided to revamp the content substantially. To add to my challenges, (and thanks to "testing" my slides before the talk in the conference room), I also realised the slide size format I was using is incorrect, and also the projector was not "setup / configured" correctly, making all my slide colours go haywire.

So after last 10 minutes of scrambling before the talk time, I managed to get this done correctly (at least that is what I think now in hindsight.

Moral of the above story - do a test / dry-run of your slides before your audience comes in!

That said, here is the abstract of the talk.

Abstract

In this age of a variety of cloud-based-services for virtual Mobile Test Labs, building a real-(mobile)-device lab for Test Automation is NOT a common thing – it is difficult, high maintenance, expensive! Yet, I had to do it! 

The slides are part of the discussion on the Why, What and How I built my own MAD-LAB (Mobile Automation Devices LAB). The discussion also includes the Automation Strategy, Tech Stack, Capabilities & Features of MAD-LAB and the learnings from successful & failed experiments in the journey. 

Slides

Below are the slides from my talk. The link to the video will be shared once available.

MAD-LAB - Capabilities And Features from Anand Bagmar

Some pictures

Agile Testing & Patterns for a good Test Automation Frameworks

2018 started with a bang! I got an opportunity to speak and share my experiences in 2 rocking meetups.

Patterns of a "good" Test Automation Framework

TechnoWise meetup on 13th Jan on Patterns of a "good" Test Automation Framework went very well, with lot of interaction and discussions along the way.

Patterns of a “good” test automation framework from Anand Bagmar

What is Agile Testing? How does Automation Help?

Then, there was an impromptu meetup setup by a very proactive ISQA community at GO-JEK office in Jakarta, Indonesia on 16th Jan. The topic there was What is Agile Testing? How does Automation help? In this meetup, I also covered aspects of Career Path of a Tester, and QA Skills and Capabilities.

There were many amazing experiences from this meetup -

• The ISQA community is very active. The meetup was setup in literally a few days and there were over 150+ attendees
• The GO-JEK office space is a very fun place. They actually have an auditorium in the office to host meetups and similar activities, apparently, once a week!!
• The questions / interactions with the attendees were very insightful

Here are slides shared in the meetup. I will share the link of the video as well once available.

What is Agile Testing? How does Automation help? from Anand Bagmar

Understanding, Measuring and Building Consumer Quality

It has been a long time since I posted anything on my blog. For those who don't know, I am working in Vuclip, a B2C company in the OTT space, where we have millions of consumers using our product via Android and iOS native apps, and the Browser too.

In the past few months, I have been in deep water taking on a new and very exciting initiative. Before I share what that is - here is a traditional approach to Quality.


Typically practices, processes, tools are chosen and implemented to help build a good Quality Product for the end-user. Evolving from Waterfall methodology to Agile methodology has been challenging for many (organizations and individuals), but has proven to be a huge step forward to achieving the goal of building a good and usable product.

In this course of time, we have (thankfully) changed the thought process of considering QA to be the "gate-keeper of Quality" to QA being a "Quality Advocate and Quality Enabler" for the team and the product. A very important change as a result has been changing the focus of QA from "finding defects" to "preventing defects".

And rightly so! After-all, why should the QA be the gate-keeper and -

• take the responsibility and blame of someone giving poor / incomplete requirements? or,
• someone writing bad code during development?

The QA is not a scavenger meant to clean up mess created by others. The QA instead is an enabler who -

• helps bring all stakeholders together through the life-cycle of the product - from conceptualization to end-delivery, 
• asks a lot of questions to find gaps, clarify assumptions, etc.
• helps find and radiate information including risks, and,
• is an active part of doing whatever it takes to prevent defects coming into the system

The Agile practices help do this in a collaborative way, getting features to completion in an incremental fashion, and iterating / pivoting based on the feedback received. This is also what practices related to Continuous Delivery enables us to do well.

But this is nothing new, at least for me. After all, during my fantastic journey at ThoughtWorks, I would say that these were basic tenets of why and how we worked.

That said, my eye-opener in the past few months has been to take this thought process many steps forward.

My agenda has been - how can I help influence and raise the bar of quality in such a way that we not only build a quality product, but also be in a position to predict how our millions of consumers will be able to use it.

This initiative we are calling as Consumer Quality -

• how do we understand Quality (= value) of the product as perceived by our Consumers, 
• what data can be relevant to understand this, how can we be proactive about looking at this data while building a quality product, and,
• the Nirvana stage - how can we predict what actions taken will have desired impact on Consumer Quality!

I hope to be able to share with you more of this in 2018!

Happy New Year everyone! Keep Learning, Keep Sharing!

Analytics - the forgotten child!

After a long time, I spoke about What, Why and How of Analytics Testing at Selenium Conference, Berlin 2017.

This talk was initially supposed to be focussed on Web Analytics only, with impact on / of IoT (Internet of Things) and Big Data, but my recent experiences made me realise, the learnings could easily be applied to Analytics from Mobile native apps as well.

So against better judgement, a full 30 minutes before I was supposed to go on stage, I started a revamp of the slides to include more content, which also meant a complete change of flow of the talk / slides. Talk about making stupid decisions, but thankfully, it turned out pretty ok!!

Abstract of the talk:

What is Web Analytics and why is it important? We'll walk through techniques for manually testing your data and automating the validation process.

Just knowing about Analytics is not sufficient for business now. There are new kids in town - IoT and Big Data - two of the most used and well-known buzz words in the software industry! With a creative mindset looking for opportunities to add value, the possibilities for IoT are infinite. With each such opportunity, there's a huge volume of data being generated which, if analysed and used correctly, can feed into creating more opportunities and increased value propositions.

There are 2 types of analysis that one needs to think about:

1. How is the end-user interacting with the product? - This will give some level of understanding into how to re-position and focus on the true value add features for the product.
2. What are the patterns in the data? - With the huge volume of data being generated by the end-user interactions, and the data being captured by all devices in the food-chain of the offering, it is important to identify patterns and find out new product and value opportunities based on these.

Video from the talk:

Slides from the talk:

The What, Why and How of Analytics Testing from Anand Bagmar

NullPointerException from RemoteWebElement in Selenium via Appium Java-Client 5.0.0-BETA9

As you may be aware from my previous posts about MAD-LAB, we are using Appium, with Java-Client 5.0.0-BETA9 to automate user journeys of the VIU app on Android & iOS devices.

Last week, suddenly, while in the middle of doing another round of significant changes to support more capability in the test framework for the Android app, the tests started failing. All infrastructure pieces were working fine, but when the App launched, I started getting this error:

ERROR AndroidLanguageScreen:16 - [5203bb1ae2771425] - ERROR in clicking on androidElement - 'By.id: tv_one' - exception - 'null'

java.lang.NullPointerException

The code in question was - driver.findByElement(myElementLocator).click()

On further investigation, it seemed that there was a problem in doing any interaction with the app, not just "click".

After lot of racking my head, asked a colleague to see if the problem reproduces on her machine. As she had not run the tests on her machine since a few days, as soon as she ran the test execution command, soon the same error happened on her machine as well. Interestingly though, we observed the following trace in her machine's console logs:

------------

Packages that were updated:

Download https://repo1.maven.org/maven2/org/seleniumhq/selenium/selenium-support/3.5.1/selenium-support-3.5.1.pom

Download https://repo1.maven.org/maven2/org/seleniumhq/selenium/selenium-api/3.5.1/selenium-api-3.5.1.pom

Download https://repo1.maven.org/maven2/com/google/guava/guava/23.0/guava-23.0.pom

Download https://repo1.maven.org/maven2/com/google/guava/guava-parent/23.0/guava-parent-23.0.pom

Download https://repo1.maven.org/maven2/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.pom

Download https://repo1.maven.org/maven2/com/google/errorprone/error_prone_annotations/2.0.18/error_prone_annotations-2.0.18.pom

Download https://repo1.maven.org/maven2/com/google/errorprone/error_prone_parent/2.0.18/error_prone_parent-2.0.18.pom

Download https://repo1.maven.org/maven2/com/google/j2objc/j2objc-annotations/1.1/j2objc-annotations-1.1.pom

Download https://repo1.maven.org/maven2/org/codehaus/mojo/animal-sniffer-annotations/1.14/animal-sniffer-annotations-1.14.pom

Download https://repo1.maven.org/maven2/org/codehaus/mojo/animal-sniffer-parent/1.14/animal-sniffer-parent-1.14.pom

Download https://repo1.maven.org/maven2/org/codehaus/mojo/mojo-parent/34/mojo-parent-34.pom

Download https://repo1.maven.org/maven2/org/codehaus/codehaus-parent/4/codehaus-parent-4.pom

Download https://repo1.maven.org/maven2/org/seleniumhq/selenium/selenium-remote-driver/3.5.1/selenium-remote-driver-3.5.1.pom

Download https://repo1.maven.org/maven2/org/seleniumhq/selenium/selenium-support/3.5.1/selenium-support-3.5.1.jar

Download https://repo1.maven.org/maven2/org/seleniumhq/selenium/selenium-api/3.5.1/selenium-api-3.5.1.jar

Download https://repo1.maven.org/maven2/com/google/guava/guava/23.0/guava-23.0.jar

Download https://repo1.maven.org/maven2/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar

Download https://repo1.maven.org/maven2/com/google/errorprone/error_prone_annotations/2.0.18/error_prone_annotations-2.0.18.jar

Download https://repo1.maven.org/maven2/com/google/j2objc/j2objc-annotations/1.1/j2objc-annotations-1.1.jar

Download https://repo1.maven.org/maven2/org/codehaus/mojo/animal-sniffer-annotations/1.14/animal-sniffer-annotations-1.14.jar

Download https://repo1.maven.org/maven2/org/seleniumhq/selenium/selenium-remote-driver/3.5.1/selenium-remote-driver-3.5.1.jar

:buildSrc:compileJava UP-TO-DATE

------------

This trace meant that something had changed in the dependencies (automatically), and gradle was fetching newer versions for the same.

This was a smoking gun we were looking for. On investigation for selenium 3.5.1 with appium java-client 5.0.0-BETA9, it quickly showed only 1 hit in search result - which was a bug reported on Java-Client 5.0.0-BETA9 - Warning: Selenium 3.5.1 breaks java client 5.0.0-BETA9

The solution / workaround was also already provided by QAutomatron

configurations.all {
    resolutionStrategy {
        force 'org.seleniumhq.selenium:selenium-support:3.4.0',
                'org.seleniumhq.selenium:selenium-api:3.4.0'
    }
}

This resolved our issue for now.

Why I needed to build my own MAD-LAB

I spoke about "Build your own MAD-LAB - for Mobile Test Automation" at vodQA - The Saga Continues! at Vuclip in collaboration with ThoughtWorks on Sat, 29th July 2017.

Join the vodQA group on facebook / LinkedIn to be part of the vodQA community.

Here are details of the talk:

Description

Building a real-(mobile)-device lab for Test Automation is NOT a common thing – it is difficult, high maintenance, expensive! Yet, I had to do it!


Setting the stage - I am coordinating all Testing activities for VIU - an OTT (over-the-top entertainment) product available on Android, iOS and WAP platforms. This product delivers high quality, popular video content in various different languages for consumers in various different regions. One of the main items in my charter is to implement functional test automation for consumer / user functionalities, and to provide quick feedback to the team and stakeholders on the “true” state of the product on all supported platforms for VIU.


In this talk, using the above set context, I will be sharing the following:

• The automation strategy
• Chosen tech-stack
• How (and why) no cloud-based solution worked for me
• Implementation details - MAD-LAB - which arose from the learnings of the failed experiments - which resulted in setting up my own real-device in-house lab.
• The core implementation (code) of MAD-LAB (already open-sourced)

Takeaways for attendees

• Learning from my experiments (what worked, or didn’t)
• Approach to testing an OTT (entertainment domain) product
• How to build a Test Automation Framework using cucumber-jvm / Appium
• Implementation details to Manage Devices, Optimizing Test Execution via distribution, Appium server, Custom Reporting etc., enabling automatic test execution via CI on each new app build, and more.

Slides

Build your own MAD-LAB - for Mobile Test Automation from Anand Bagmar

Video (talk starts at 04m:45s)

https://www.youtube.com/watch?v=yiRbfhOqQuc#t=04m45s