Getting Selenium 4's DevTools working with RemoteWebDriver & Docker

I was facing different errors in using Selenium 4's DevTools working with the Chrome browser in Docker container.

Thanks to Titus Fortner and Diego Molina's quick help, I was able to resolve the same. You can see the conversation with the awesome Selenium Committers here.

In case you are encountering the same problem, see my github repo (https://github.com/anandbagmar/NetworkInterceptor) for a working sample of how to set it up.

Speaking schedule in the past year, and crazy July 2023!

I am writing on my blog after a very long time ... Don't remember the last time I wrote a post. So what has happened in the past 6-12 months?

Thanks to the wonderful conference and meetup organizers, I have traveled a lot for the following:

• DevOps Days 2022 @Switzerland
• Meetup @New York
• Meetup @Boston
• Selenium Conf 2023 @Chicago,
• Meetup @Sydney,
• Testing Talks 2022, Melbourne
• StarEast 2023, Orlando

I also spoke at many meetups and events in Gurgaon, Hyderabad, Bangalore, Mumbai and Pune.

I also spoke in some very interesting webinars in this time. The ones that surprised me the most because of the huge popularity and turnouts were:

• Unlocking the Power of ChatGPT and AI in Testing: A Real-World Look
• Unlocking the Power of ChatGPT and AI in Test Automation: Next Steps

While doing this, I have been contributing to open-source - teswiz and AppiumTestDistribution are the big noteworthy items. Recent work has been to support Selenium WebDriver v4 and Appium 2.0 in these frameworks.

Now in July 2023, this is the scheduled lineup of events (with dates, location and topics):

• Saturday, 8 July 2023 - A modern look at AI-powered Cross Browser Testing at Game of Testing, Bangalore
• Thursday, 13th July 2023 - Can AI Autogenerate and Run Automated Tests? - An Applitools webinar.
• Monday, 17th July 2023 - Visual Validation - The missing tip of the automation pyramid & and A modern look at AI-powered Cross-Browser Testing at Gentrack Testing Meet Up, Auckland, NZ.
• Thursday, 20th July 2023 - Unlocking the Power of ChatGPT and AI in Testing & Test Automation, at the Software Test Automation Group (STAG) meetup, Melbourne, Australia.
• Monday, 24th July 2023 - Eradicate Flaky Tests and Automating the real-user scenarios across multi-apps, and multi-devices, at All Things Testing Brisbane, Australia.
• Wednesday, 26th July 2023 - Power your Automation using AI, at Testing Talks Sydney Australia.
• Thursday, 27th July 2023 - Metrics to measure the effectiveness of your engineering processes, practices, and implementation, at ITMA-Applitools event, Singapore.

Automating the real-user scenarios across multi-apps, and multi-devices

Simulating real-user scenarios as part of your automation is a solved problem. You need to understand the domain, the product, the user, and then define and implement your scenario.

But there are some types of scenarios that are complex to implement. These are the real-world scenarios having multiple personas (users) interacting with each other to use some business functionalities. These personas may be on the same platform or different (web / mobile-web / native apps / desktop applications).

Example scenarios:

• How do you check if more than 1 person is able to join a zoom / teams meeting? And that they can interact with each other?
• How do you check if the end-2-end scenario that involves multiple users, across multiple apps works as expected?
  • Given user places order on Amazon (app / browser)
  • When delivery agent delivers the order (using Delivery app)
  • Then user can see the order status as "Delivered"

Even though we will automate and test each application in such interactions independently, or test each persona scenarios independently, we need a way to build confidence that these multiple personas and applications can work together. These scenarios are critical to automate!

teswiz, an open-source framework can easily automate these multi-user, multi-app, multi-device scenarios. 

Example: Multi-user, Multi-device test scenario

 

Example: Multi-user, Multi-app, Multi-device test scenario

See teswiz and getting-started-with-teswiz projects for information, or contact me.
 

How to handle Select certificate popup from Chrome using Selenium WebDriver?

For one of the applications, when I run some Selenium WebDriver tests. I see a Select a certificate popup. I am not able to handle this, and hence my test fails. 

Has anyone been able to handle this popup, or rather, avoid this popup being shown?

If I manually select a certificate and click OK, I am prompted for my mac credentials:

Thanks in advance!

Why not to use PageFactory and FindsBy in Selenium WebDriver

Many users of Selenium WebDriver may be using the PageFactory created by Simon Stewart. However, it is not a good idea to use it.

You may be thinking why should I not use it? It is so easy to use it, and its popular.

Well, here are 2 reasons why you should not use the PageFactory:

Reason #1. Simon Stewart (https://twitter.com/shs96c), the creator of WebDriver, and the PageFactory himself says, do not use it. It is not recommended.

The `FindsBy` annotation isn't recommended, because the PageFactory class is really badly implemented and inflexible, but it's not going away in the java bindings.

The `FindsByX` interfaces are going away. Better to use a `By` locator and use that.

 

https://twitter.com/shs96c/status/1196865907185868801

Reason #2: While Reason #1 should have been sufficient, many people implementing automation using Selenium WebDriver do not know, or did not pay heed to what Simon said. So another WebDriver & WATIR contributor, Titus Fortner (https://twitter.com/titusfortner) explained in detail why using PageFactory is not a good idea in his blog post - https://titusfortner.com/2021/02/03/page-factory-optimization.html. 

 

I sincerely hope these reasons are sufficient for you to move away from the PageFactory and use something more efficient. 

 

Automating Functional / End-2-End Tests Across Multiple Platforms

Do you want to know more about an approach to Functional (end-to-end) Test Automation that works for a product available on multiple platforms?

If yes, then read my blog post on Automating Functional / End-2-End Tests Across Multiple Platforms which shares details on the thought process & criteria involved in creating a solution that includes how to write the tests, and run it across the multiple platforms without any code change.

Lastly, the open-sourced solution - teswiz also has examples on how to implement a test that orchestrates the simulation between multiple devices / browsers to simulate multiple users interacting with each other as part of the same test.

Long time no see? Where have I been

It has been a few months since I published anything on my blog. Does not mean I have not been learning or experimenting with new ideas. In fact, in the past few months I have been privileged to have my articles published on Applitools and Test Project blogs.

Below is the link to all those articles, for which I have received very kind reviews and comments on LinkedIn and Twitter.

Apart from this, I have also been contributing to open source - namely - Selenium, AppiumTestDistribution and building an open-source kickstarter project for API testing using Karate and for end-2-end testing for Android, iOS, Windows, Mac & Web.

Lastly, I have also been speaking in virtual conferences, webinars and last week I also recorded a podcast, which will be available soon.

The end of Smoke, Sanity and Regression

https://applitools.com/blog/end-smoke-sanity-regression/

Do we need Smoke, Sanity, Regression suites?

• Do not blindly start with classifying your tests in different categories. Challenge yourself to do better!
• Have a Test Automation strategy and know your test automation framework objective & criteria (“Test Automation in the World of AI & ML” highlights various criteria to be considered to build a good Test Automation Framework)
• Choose the toolset wisely
• After all the correct (subjective) approaches taken, if your test execution (in a single browser) is still taking more than say, 10 min for execution, then you can run your tests in parallel, and subsequently, split the test suite into smaller suites which can give you progressive indication of quality
• Applitools with its AI-power algorithms can make your functional tests lean, simple, robust and includes UI / UX validation
• Applitools Ultrafast Grid will remove the need for Cross-Browser testing, and instead with a single test execution run, validate functionality & UI / Visual rendering for all supported Browsers & Viewports

Design Patterns in Test Automation

https://blog.testproject.io/2020/06/29/design-patterns-in-test-automation/
 

Criteria for building a Test Automation Framework

Writing code is easy, but writing good code is not as easy. Here are the reasons why I say this:

• “Good” is subjective.
• “Good” depends on the context & overall objective.

Similarly, implementing automated test cases is easy (as seen from the getting started example shared earlier). However, scaling this up to be able to implement and run a huge number of tests quickly and efficiently, against an evolving product is not easy!

I refer to a few principles when building a Test Automation Framework. They are:

• Based on the context & (current + upcoming) functionality of your product-under-test, define the overall objective of Automation Testing.
• Based on the objective defined above, determine the criteria and requirements from your Test Automation Framework. Refer to my post on “Test Automation in the World of AI & ML” for details on various aspects you need to consider to build a robust Test Automation Framework. Also, you might find these articles interesting to learn how to select the best tool for your requirements:
• Criteria for Selecting the Right Functional Testing Tools
• How to Select the Best Tool – Research Process
• How To Select The Right Test Automation Tool

Stop the Retries in Tests & Reruns of Failing Tests

https://applitools.com/blog/uncover-flaky-tests/
 

Takeaways

• Recognise reasons why tests could be flaky / intermittent
• Critique band-aid approach to fixing flakiness in tests
• Discuss techniques to identify reasons for test flakiness
• Fix the root-cause, not the symptoms to make your tests stable, robust and scalable!

Measuring Code Coverage from API Workflow & Functional UI Tests

https://blog.testproject.io/2020/11/24/measuring-code-coverage-from-api-workflow-functional-ui-tests/

 
Why is the Functional Coverage important?

I choose an approach keeping the 80-20 rule in mind. The information the report provides should be sufficient to understand the current state, and take decisions on “what’s next”. For areas that need additional clarity, I can then talk with the team, explore the code to get to the next level of details. This makes it a very collaborative way of working, and joint-ownership of quality! 🚀

You can choose your own way to implement Functional Coverage – based on your context of team, skills, capability, tech-stack, etc.

Overcoming chromedriver version compatibility issues the right way

I encountered an interesting challenge recently when doing Native Android / iOS app automation - this was related to Chrome browser versions getting updated automatically and my tests failing because of errors like:

org.openqa.selenium.SessionNotCreatedException: session not created: This version of ChromeDriver only supports Chrome version 74 23:04:25 (Driver info: chromedriver=74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729@{#29}),platform=Windows NT 6.3.9600 x86_64) (WARNING: The server did not provide any stacktrace information)

So I asked a question on LinkedIn


And I tweeted asking how to manage ChromeDriver version when running WebDriver / Appium tests.

I have a #appium / #java #framework.
What is a good way to keep #chromedriver updated with version of #chrome via #scripts? I do not want to do this manually - since I have 3+ #jenkins agents (#linux + #mac mini) and 7+ team members who need to keep doing this #osx #command

— Anand Bagmar (@BagmarAnand) September 20, 2019

The answer was common and obvious – use WebDriverManager. This is a beautiful, simple and indeed the right answer and solution to the problem.

However, that was a partial answer for me. 

Here is my context and problem statement in detail:

• My Test Automation Framework is based on Java / Appium and I use AppiumTestDistribution (ATD) 
• ATD is open-source, and takes away my pain and effort of managing appium and the devices and also takes care of running the tests in parallel or distributed mode, on android as well as iOS
• In my local lab setup, I have many different android devices connected - which run tests as directed by ATD
• Since you cannot control how Google PlayStore / Apple App Store pushes out new versions of apps for different android / iOS versions on devices, it is easily possible to end up with different versions of chrome browser in your device lab. When this happens, the tests start failing because of chromedriver incompatibility issues.

Once I was very kindly reminded by the community about WebDriverManager (which I had forgotten about), I now knew what was to be done.

I looked at the ATD code, and realised that it was using the default chromedriver version as setup when I had installed appium. This chromedriver was being used when instantiating a new instance of the AndroidDriver.

So I submitted a PR for ATD - which essentially did the following:

• Query the chrome browser versions on each connected device
• For the **highest version of the browser, use WebDriverManager and get the appropriate chromedriver downloaded
• Pass the path to the correct chromedriver when creating an instance of the AndroidDriver

**highest version - what does that mean? Well, I also got confused initially. But the answer was simple. On some devices, the Chrome browser is installed by default, as a system app. This cannot be removed. So as new versions of the browser get installed, the default Chrome system app is always there. So when you query for the versions of Chrome on the device, you will see 2 such versions. My code logic was to get all these versions, and pick the highest version from them.

Here is the code snippet of how I solved the problem:

Special thanks to Sai Krishna for quickly approving and merging this PR.

Hope this provides more information about my problem statement, and how I used your suggestion for WebDriverManager to solve the problem.

Analytics - The Brain of the Software

An Analogy 

I am not a doctor, nor did I enjoy biology too much in my curriculum as a student.  However, I do know that the body has many organs and each organ plays a vital role in the well being of the individual.

Each organ has to:

• function correctly (movement, senses, core functions, etc.)

• has to perform as per expectations in different conditions the individual may be going through (walking, running, swimming, etc.)

• has to be secure from external parameters (heat, cold, rain, what we eat / drink, etc.)

• has to have a proper user experience (ex: if the human hands had webs like ducks, would we be able to hold a pen correctly to write?

• I would like to think of the brain as the super computer which keeps track of what is going on in the body, if each piece playing its part correctly, or not. And if there is something unexpected going on, then there are mechanisms of giving that feedback internally and externally so that course correction would be possible.

How does this relate to software?

Software is similar in some ways. For any software product to work, the following needs to be done:

Functionality works as expected

• The architecture, testability of the system will allow for various types of testing activities to be performed on the software to ensure everything works as expected
• Test Automation practices will give you quick feedback

There is a plethora of open-source and commercial tools in this space to help in this regard - the most popular open-source tools being Selenium and Appium.

Software is performant

• We can do performance testing at various different levels to ensure at different loads and conditions, the users will be able to use the product in a seamless fashion
• There are many tools to assist in Performance Testing - some popular ones being JMeter and Gatling.

Software is secure

• Building and testing for security is critical as you do not want user information to be leaked or manipulated and neither do you want to allow external forces to control / manipulate your product behaviour and control
• The Test Automation Pyramid hence also includes NFRs

User experience is validated, and consistent

• In the age of CD (Continuous Delivery & Continuous Deployment), you need to ensure your user experience across all your software delivery means (browsers, mobile-browsers, native apps for mobiles and tablets, etc.) is consistent and users do not face the brunt of UI / look-and-feel issues in the software at the cost of new features
• This is a relatively new domain - but there are already many tools to help in this spaces as well - the most popular one (in terms of integration, usage and features) being the AI-powered Applitools

Visual Validation is the new tip of the Test Automation Pyramid!

What is the brain of the software?

The above is all good, and known in various ways. But what is the "brain" of the software? How does one know if everything is working fine or not? Who will receive the feedback and how do we take corrective action on this?

Analytics is that piece in the Software product that functions as the brain. It keeps collecting data about each important piece of software, and provides feedback on the same.

I have come across some extreme examples of Business / Organizations who have all their eggs in one basket - in terms of

• understand their Consumers (engagement / usage / patterns / etc.),
• understand usage of product features, and,
• do all revenue-related book-keeping

This is all done purely on Analytics! Hence, to say “Business runs on Analytics, and it may be OK for some product / user features to not work correctly, but Analytics should always work” - is not a myth!

What this means is Analytics is more important now, than before.

Unfortunately, Analytics is not known much to the Software Dev + Test community. We know it very superficially - and do what is required to implement it and quickly test it out. But what is analytics? Why is it important? What is the impact of this not working well? Not many think about this.

I have been testing Analytics since 2010 ... and the kind of insights I have been able to get about the product have been huge! I have been able to contribute back to the team and help build better quality software as a result.

But I have to be honest - it is painful to test Analytics. And that is why I created an open-source framework - WAAT - to help automate some of this testing activities.

I also do workshops to help people learn more about Analytics, its importance, and how they can automate this as well.

In the workshop, I do not assume anything and approach is to discuss and learn by example and practice, the following

• How does Analytics works (for Web and Mobile)?
• Test Analytics manually in different ways
• Test Analytics via the final reports
• Why some Automation strategies will work, and some WILL NOT WORK (based on my experience)!
• We will see a demo of the Automation running for the same.
• Time permitting, we will setup running some Automation scripts on your machine to validate the same

Takeaways from the workshop

We will learn by practicing the following:

• What is Analytics?
• Techniques to test analytics manually.
• How to automate the validation of analytics, via a demo, and if time permits, run the automation from your machine as well.

Hope this post helps you understand the importance of Analytics and why you need to know more about it. Do reach out to me if you want to learn more about it.

Next upcoming Analytics workshop is in TestBash Australia 2019. Let me know if you would be interested in attending the same

Test Automation in the World of AI and ML

My article on "Test Automation in the World of AI & ML" recently got published on InfoQ.


Here are the key takeaways mentioned in the article -

• There are many criteria to be considered before building framework / selecting tools for Functional Test Automation
• It is very important to prioritise framework / tools capabilities needed for the software-under-test
• A good, scalable Test Automation Framework that provides fast and reliable feedback to the team enables collaboration and CI/CD
• Debugging / RCA (root cause analysis) and support for libraries / tools used is an afterthought in most cases. Do not fall in that trap.
• There are some promising commercial tools that fit seamlessly in the Agile way of working. Depending on the complete context, these tools may be a good choice over building your own framework for Functional Automation.

You can read the full article from here. 

Looking forward to comments on the same!

.

Testing in the Agile World

Thanks to ThoughtWorks, I was introduced to many things - 

• Agile
• Blogging
• Speaking at conferences
• Contributing to Open-source
• ...
• ...

The list is actually quite long - but that is not the intention of this post.

The main takeaway in my learning at ThoughtWorks though, is how to Test better, and be more effective in that for the end-user. 

Even before my time at ThoughtWorks, I never agreed with the thought process that Functional Automation can / should be done only when the feature is stable. But at ThoughtWorks, I did learn many more tips and tricks and techniques and processes how to do this Functional Automation in a better way, as the product is evolving.

On 9th April 2011, I had written a detailed blog post / article regarding how can we test better in the Agile world. 

This post was titled - "Agile QA Process", and the document was uploaded to slideshare with the name - "Agile QA Process". I am very pleasantly surprised that till date, that document has had over 74K views and almost 2.7K downloads, and is still my topmost viewed post on slideshare.

When I look back at the document, it still seems very relevant and applicable, to me! 

What do you think?

Converting JSON into usable objects

JSON is a great way to specify data / information and, off late, it is the format of my choice to specify test data.


I find it to be -

• light weight 
• easy to understand 
• almost very intuitive to know if you have made an error in the syntax 
• easy to read into code and parse 
• easy create meaningful custom objects and use in code 

Recently, thanks to a friend - Abhijeet Vaikar, I came across a tool - quicktype.io - that helps in transforming the raw JSON (from various sources) directly into custom objects, in a variety of languages.

Site: https://quicktype.io/

The tool: https://app.quicktype.io/

I got to know about this tool at perfect time as I am building a new tool for dynamic logging in Java - AutoLogJ (but more about AutoLogJ later). quicktype.io does what it promises - and it saved me a lot of time to build the custom POJOs for the same.

Thanks Abhijeet Vaikar and the quicktype team!

Implementing Soft Assertions

Back in 2009 / 2010, I was working on implementing end-2-end tests for a web site using Java / Selenium / TestNG based automation. The challenge I was facing was that the tests used to fail for trivial (but valid) reasons - and I wondered that the test did not even get to core validations before it failed. How will the team ever know about the main issues in the product if the test fails for trivial issues? 

That was a trigger point for me to think about Soft Assertions - what if there was a way to say if there is a type of failure that I want to know about, but the test can continue with the remaining set of validations - unless something does not make sense to proceed with.

Ex: If the text message of a field is incorrect, I can continue. But if login fails, no point in proceeding with the rest of the test.

This idea seemed interesting - so I came up with the following requirements from such an  implementation as listed below:

• Clear distinction between what type of failure I can continue from, or not
• Ex: assert.** is for hard asserts. verify.** is for soft asserts
• All failures that I can continue from (i.e. soft asserts), need to be collated and at the end of the test, the complete list of those soft assert failures should be presented with the test result (and in the report), while the test failed just once
• Ex: There were 5 soft assertion failures)
• Capture relevant screenshots whenever Soft Assertion failed
• If there was a hard assert along the way of the test execution, the test failure should include the prior soft assert failures along with the hard assert failure, as appropriate

For the actual implementation, I did the following (in 2009/2010):

• I looked into the TestNG code base, and I could not really find any out-of-the-box support for what I wanted to do. 
• So for lack of knowledge on better ways of implementation, 
• I checked-out the TestNG code, 
• added the Soft Assertion implementation, and, 
• built a custom TestNG.jar file
• checked-in the jar file as a library artefact in our automation framework. 

In hindsight, I should have sent that functionality as a PR to the project. 

But not all is lost, TestNG now (or maybe since some time now) has support for Soft Assertions - out-of-the-box. And it is pretty straightforward to implement / use it as well.

Implementing Soft Assertions in your test framework

See this gist for implementation that you can you use with TestNG (I tested with v6.10).

Using Soft Assertions in your tests

Here is how you can use the Soft Assertions in your tests.

Soft Assertions in any other tech stack?

What if you are not using TestNG, or Java - rather, what if you are using completely different programming language / tools / test-runner? Can you still use Soft Assertions? 

Absolutely YES! All you need to understand is the concept, and figure out the best way to implement the same, if any out-of-the-box solution does not exist in that tech stack. 

Hope this helps you!

Return of the todo (learning) list

Since at least a decade, I have had a list of TODOs which I actively updated and maintained. The items on this list focused on - 

• what new things I wanted to learn / experiment with
• new conference talk ideas
• open source ideas / updates

Unfortunately, due to various reasons (some of which were my doing as well), the focus on learning and experimentation got lost ... and I felt awful about it. I also shared my pain with some friends and colleagues of unable to find time / opportunity / focus / support to be in the continuous learning phase. But not much could be done / changed in those circumstances.

But I am very happy to share that the list is back, and back with a bang!!!

The days of learning and experimentation continue. My list is now overflowing, and continuing to grow with ideas and things I want to learn and experiment.

I am happy again!!

Essence of Testing - A new beginning

In my career so far, I have been very fortunate to have got great mentors, and a variety of opportunities to learn, add value, and share my experiences with others around me.

Here are some of these experiences:

• Worked in various sized organisations across the globe in the past couple of decades
• The teams have been big and small
• Played a variety of roles - Quality Analyst, SDET (Software Development Engineer in Test), Product Quality Engineer (PQE), Automation Engineer, Consultant, Coach, Project Manager, Director - Quality, Support Engineer, etc.
• Worked with teams having products in different domains - Health care, eCommerce, Banking / Finance, Retail, Entertainment (OTT), Research, etc. 
• With in organisations (B2B and B2C space): WebMD, Borland Software, Microsoft (Redmond, USA), AmberPoint (USA / India), ThoughtWorks, Vuclip 
• Shared experiences with other via Meetups and Conferences world-wide
• Created opensource tools like WAAT, TTA, TaaS

After working as an employee in these organisations for almost a couple of decades now, I have taken a plunge to do something different.

I am now looking forward to work with Organisations and Teams, to help co-create optimized solutions towards shipping a quality product. Leave me a message on my blog, or send me an email at abagmar@gmail.com to talk more on how we can work together!

As part of this journey, even before I was able to buy my own laptop, and warm it up, I got an interesting consulting assignment, thanks to a dear friend.

This assignment was exactly what I needed to get started - it was a Discovery Workshop, with the following objective:

• Learn and understand the current state of the team and the product they were building
• Understand current (perceived) challenges
• Suggest improvements for the team in areas of:
• Tweaks in the current processes
• Practices to be adopted / got better at / or stopped (as anti-patterns and not adding value)
• Identify opportunities to Test better, and early
• Suggest a Test Automation Pyramid that is fit-for-purpose for the team
• Suggest strategy, tools and approach for end-2-end (e2e) functional automation

As I usually do, I started off the workshop with my favorite tool for note taking - Mind Maps! As the conversations evolved and got specific, I started with a Balanced Mind-Map  to Fishbone Mind-Map.

Eventually I consolidated my thoughts and created the Discovery Workshop Report for the client in the form of a slide deck, using the following approach.

• Learning from Discovery
• Talk with the Team members
• See & understand the project management tool, quality of requirements, etc.
• See the code - understand complexity, types of tests written, quality of tests, etc.
• See the Testing related artifacts - test cases, test execution strategy, exploratory testing, etc.
• See the CI server - how deployments happen, what causes the builds to fail, etc.
• Recommendations
• For all the above areas, I created recommendations on what different aspects may help the team move ahead in a better way

Discovery and Recommendations were based on each specific activity:

• Process
• Architecture
• Requirements
• Development
• Testing
• Deployments

Once the learning from the Learning and Recommendation stages were well understood (by me), I then created a Suggested Plan of Execution (in phases / milestones)

• From the recommendations, I created milestone based plans on what can be started immediately, and what decisions the team needs to take to move forward in other areas

In this 2 week time, unknowingly, in retrospect, the whole engagement was quite agile. There were periodic demos of the POCs, regular progress sharing, and changing direction of discovery and recommendations based on learning.

The Client also appreciated the quality of conversations, and the results that were shared.

All in all, a very satisfying beginning to my new journey with Essence of Testing!

MAD-LAB - Capabilities & Features - Agile India 2018

I spoke about "Build your own MAD-LAB - for Mobile Test Automation for CD" at Agile India 2018.

Though I have spoken on this similar topic answering the question - "Why I needed to build my own MAD-LAB?" before at vodQA in July 2017 at Vuclip, quite a few things have changed since then.

Knowing the value of "being agile", a day before my scheduled talk in Agile India 2018, I decided to revamp the content substantially. To add to my challenges, (and thanks to "testing" my slides before the talk in the conference room), I also realised the slide size format I was using is incorrect, and also the projector was not "setup / configured" correctly, making all my slide colours go haywire.

So after last 10 minutes of scrambling before the talk time, I managed to get this done correctly (at least that is what I think now in hindsight.

Moral of the above story - do a test / dry-run of your slides before your audience comes in!

That said, here is the abstract of the talk.

Abstract

In this age of a variety of cloud-based-services for virtual Mobile Test Labs, building a real-(mobile)-device lab for Test Automation is NOT a common thing – it is difficult, high maintenance, expensive! Yet, I had to do it! 

The slides are part of the discussion on the Why, What and How I built my own MAD-LAB (Mobile Automation Devices LAB). The discussion also includes the Automation Strategy, Tech Stack, Capabilities & Features of MAD-LAB and the learnings from successful & failed experiments in the journey. 

Slides

Below are the slides from my talk. The link to the video will be shared once available.

MAD-LAB - Capabilities And Features from Anand Bagmar

Some pictures

Why I needed to build my own MAD-LAB

I spoke about "Build your own MAD-LAB - for Mobile Test Automation" at vodQA - The Saga Continues! at Vuclip in collaboration with ThoughtWorks on Sat, 29th July 2017.

Join the vodQA group on facebook / LinkedIn to be part of the vodQA community.

Here are details of the talk:

Description

Building a real-(mobile)-device lab for Test Automation is NOT a common thing – it is difficult, high maintenance, expensive! Yet, I had to do it!


Setting the stage - I am coordinating all Testing activities for VIU - an OTT (over-the-top entertainment) product available on Android, iOS and WAP platforms. This product delivers high quality, popular video content in various different languages for consumers in various different regions. One of the main items in my charter is to implement functional test automation for consumer / user functionalities, and to provide quick feedback to the team and stakeholders on the “true” state of the product on all supported platforms for VIU.


In this talk, using the above set context, I will be sharing the following:

• The automation strategy
• Chosen tech-stack
• How (and why) no cloud-based solution worked for me
• Implementation details - MAD-LAB - which arose from the learnings of the failed experiments - which resulted in setting up my own real-device in-house lab.
• The core implementation (code) of MAD-LAB (already open-sourced)

Takeaways for attendees

• Learning from my experiments (what worked, or didn’t)
• Approach to testing an OTT (entertainment domain) product
• How to build a Test Automation Framework using cucumber-jvm / Appium
• Implementation details to Manage Devices, Optimizing Test Execution via distribution, Appium server, Custom Reporting etc., enabling automatic test execution via CI on each new app build, and more.

Slides

Build your own MAD-LAB - for Mobile Test Automation from Anand Bagmar

Video (talk starts at 04m:45s)

https://www.youtube.com/watch?v=yiRbfhOqQuc#t=04m45s