How to enable seamless running of appium tests on developer machines?

I am implementing cucumber-jvm based framework to drive mobile apps (using Appium).

Here is what I need to be able to do -

1. Run tests on local machine for quick validation. This is mainly for developers to be able to run the tests before pushing code changes in git.
2. Trigger and Run the tests in the cloud to run against emulators / real devices. 

To achieve point #1, I need the setup to be simple. I do not want the team to go through massive steps to get the environment (Appium, emulators, etc.) setup.

Can / should the whole setup be put inside a docker container - and provide single command to setup and run the tests?

Any other approach you recommend?

Of course - whatever approach is taken, should potentially extend seamlessly to address point #2.

Career Path of a Tester!

PS: This is a long post. You can can also get / read a pdf version of this from here via slideshare.

I asked in various forums - “Does a Tester need a career path? Yes / No - Why?”

Thanks to all who replied and have shared deeper insights into different perspectives.

I agree with most of the thoughts and also think that the answer to the question is "It Depends!"

On what you ask? On various parameters being used to reach the answer. The parameters could be:

• Your past experiences
• Your current work - and how much you are enjoying it, or not. More important (IMHO is the activities you have been done, not the titles / roles you have been playing)
• Who has inspired you in the past and present - and what was their role at the time

Career Path options for a Tester

Before I proceed further, I want to share my perspective on what I think are the options for “testers” in the Software Industry. In terms of Career Path options - I think the options are in a way clear as depicted in the mind map below.

Career Path for a Tester

Note - this article focusses on all aspects except the Managerial Track

Capabilities and Skills for a Tester

Moving along the Career Path is almost natural as you spend more time in the industry. However, to become successful along the way, there is a lot of hard work required.

There are various capabilities and skills that a Tester needs to be successful in his / her career - including but not limited to the few options shown below. 

Note:

• These skills are more from a Individual Contributor / Technical Track perspective.
• The mind map does not do justice to visualize easily the Levels of expertise that are possible for each of the items. The levels can be categorised into
• Don't Know
• Beginner
• Intermediate
• Expert
• Master

Organization Focus

The Capability and Skills would differ "slightly" based on the nature / operational model of the organization you are working for.

Coaching and Consulting Focus

Coaching and Consulting focussed-roles would predominantly require strong soft-skills, but the importance and value of Technical Understanding cannot be underestimated. 

Often unfortunately, I have come across 'consultants' who know only theory, and very little practical experience of implementation. In such cases, you care not doing justice to your role. 

At the same time, I have come across brilliant Consultants who bring in a wealth of experience and knowledge (what has worked, and not worked) and apply it in the correct way in the present context. It was indeed a pleasure to work with such Consultants!

Analysis Focus

The path of the shift-left is extremely important and also not easy. You need to apply a different mindset to accomplish activities that helps build a better software even before any code is written.

Test Analyst Focus

A Tester cannot be complete without doing well in the core skills - Testing! 

Testing is complex, frustrating, tedious, slow, negative to a certain extent - finding mismatches in actual Vs expectations and can never say its 100% complete! Yet, it is a lot of fun. You get to learn a lot of things, use a variety of tools to help your Testing efforts. 

When the software is built right, and the end-user is able to do what is expected from the software you have helped build and test, Testing is also very rewarding!

There are a few things that limit the impact of Testing - 

• Lack of curiosity
• Lack of willingness to learn, explore and get to the deep core of the product
• Lack of willingness to be technical (understand / contribute to product architecture, read and understand code and the automated tests)
• Lack of willingness to experiment
• Lack of willingness to pair with different roles - Product Owners, Business Analysts, Developers, etc
• Unable to work as ONE TEAM!
• ....

Technical Focus

Software Development is a highly technical activity. There is no reason why Testing should be considered as a non-technical activity.

Incubation and Initiatives

There are other activities that a Testing can be part of - these are related to Learning, Sharing, Mentoring and Contributing to make the Testing space more richer!

The Pendulum Visualization!

Another way to visualize these skills and capabilities is to think of this as a pendulum.

• The arc created by the swinging pendulum contains all the items listed above
• As your experience in the industry grows, you should get some level of experience in each of these aspects
• Your passion and interests should determine what area(s) you want to get a deep-level of expertise in
• So eventually you will end up in a T-model - the short horizontal line representing the breadth of coverage across all aspects of how you can help build a 'good' quality product, and the vertical line representing the 'deep' level of mastery and expertise you have.

Summary

The sad part!

Now that this is out of the bag - I want to highlight a few things that I have observed in the Software industry, and this is probably not limited to just the Tester role.

• People are impatient and quickly want to move up the chain along the career path track. Motivation could be, but not limited to - Money and Power (to control). I would ask these people to do a serious introspection to figure out if they are truly doing justice to their role and their peers, Are they able to be good role models for the people “reporting” to them?
• Career Path is necessary, but overrated. Too much emphasis is given on the “role” and not enough on the activities required to make the team successful, and make the product that will help the end-user. Focus on the big-picture, and your career will grow automatically.
• We try to copy what others in the industry have done in their career, This is an easy way out. Instead, I recommend you understand their journey, the hard work your role-models have put in their life to get to where they are. I encourage each of you to spend some time understanding this, and then work with your mentors and people you trust to see how you can create your own path. That way, your life, journey and destination will be unique!

The Happy Ending!

Testing is a lot of fun. I have been in the Testing field since quite some time. Here are some things (in no particular order) that have kept me interested in the field:

• Understand the domain
• Always keep learning (domains, technologies, tools, practices, soft-skills, ...)
• Experiment - that is the best way to learn
• Don’t be afraid of failure. Learn from the mistakes, and ensure you do whatever possible to avoid the same mistake again
• Be curious
• Ask “relevant” questions
• Challenge status-quo
• Do NOT assume
• Understand risk of functionality not working - will help in proper prioritization
• “Why” before “How?”
• Know the end-users of the product-under-test. Will help test better
• Testing is NOT only about finding / reporting defects.
• Use metrics that help make meaningful decisions to make the product better
• Be patient!
• Help build a “ONE TEAM” culture!

Does a Tester need a Career Path?

On Nov 22nd, 2016, I asked a question via Twitter and Facebook - 

Does a Tester need a career path? 

• If yes, what would the options be?
• If no, why not? 

Below are the comments I received. Do you have any more thoughts on this?

A new beginning - entertainment on mobile

After 7+ years, I finally took the heavy step and moved out of ThoughtWorks.

The past 7+ years have been awesome. I had loads of fun, learnt many new things, made a lot of friends and found inspiration and guidance from a lot of mentors.

Thank you ThoughtWorks and ThoughtWorkers! Wouldn’t have been who I am today without you and you all will always be a huge part of me!

Taking the decision was tougher than I thought it would be ... but new challenges were waiting for me, and the time had come.

On 1st December, 2016, I started my next stint as "Directory - Quality" at Vuclip, Inc for the Viu product. You can also find us via the PlayStore or AppStore.

Day 1 at vuclip, barring the first 2 hours of paperwork, was getting right into action. With the planning for 2017 in full swing, there was no time to settle - but instead had to hit the ground running.

The charter starting Day 1 for me was:

• Define & execute test strategy for Viu - for multiple platforms, for multiple regions & partners
• Build team to help execute the above (see section below on what I am looking out for)
• In scope - functional testing, automation, performance, analytics, benchmarking, infrastructure, tooling, etc.
• Out of scope - nothing

And so the fun has begun.

So, here is what I need to learn and execute immediately (looking forward to suggestions, links, feedback on how you have done it in the past)

• Is it worth setting up a mobile lab (real devices + simulators) in-house or use external services for running automated tests  exploratory tests? 
• If the latter to the above, what services have you used in the past? What have the results been?
• Is it possible (and worth) automating the checks for memory / processor / battery usage when running tests against the native app (on Android & iOS)?
• How to do native app performance testing (client-side) for Android & iOS?

Also, I am looking to build a strong testing team with team members having the primary skills & capabilities -

• Open-minded, quick learner
• A good Testing-mindset
• Mobile Testing experience (non-automated + test automation)
• Performance Testing (client-side & server-side)

Contact me if you are interested in being part of my team to work on this challenging product.

Shared (relatively less) pain of using Protractor in SeConf London 2016

On 15th Nov 2016, I spoke in Selenium Conference London 2016 on the topic - "Sharing the 'pain' of using Protractor & WebDriver". This time, the pain was less - as I had already shared this in Selenium Conference India 2016. From that experience, and lot of interactions, I worked on some of the challenges I was facing and implemented good solutions for the same.

So this time it was easier.

The interesting challenge though was when the A/V stopped working after the 1st slide - and had to do on-the-spot improv to keep the ball rolling and prevent delays in subsequent talks. To know more about the fun I had, watch the video linked below :)

You will find the abstract, slides and video of the talk:

Abstract

There is a saying ..."Sukh baatne se badhta hai, dukh baatne se kam hota hai", translated as - "happiness increases & sadness reduces on sharing with others".

We want to take this opportunity to share with our experiences - the good and the bad, in the journey of building a Test Automation framework for an AngularJS based application.

We will learn, by a case study, what thought process we applied on the given context (product, team, skills, capabilities, long term vision) to come up with an appropriate Test Automation Strategy. This Test Automation strategy covered all aspects of Test Automation - Unit, Integration, UI - i.e. End-2-End tests (E2E).

Next, we will share how we went about narrowing-down, and eventually selecting a specific Tech Stack + Tools (Javascript / Jasmine / Protractor / Selenium-WebDriver) to accomplish the Test Automation for the product.
 
Lastly, we will share the challenges that came up in the implementation of the Test Automation, and how we overcame them. This will also include how we managed to get the tests running in Jenkins - a Continuous Integration tool.

This discussion is applicable to all team members who are working on Test Automation!

P.S. We will be attempting to make a sample protractor-based automation framework available on github for anyone to use as a starting point for setting up protractor-based Test Automation framework.

Slides

Sharing (less) Pain of using Protractor & WebDriver from Anand Bagmar

Video

https://youtu.be/YpEC3rFkH5

 

Recording video of test execution via Protractor

Recently I came across a use case of recording the video of the test execution in my CI environment - where tests run in headless mode). 

My thought was - record video of the test execution in CI - save it if test failed, else discard it.

Below are some resources I came across for potentially implementing this - 

• https://www.npmjs.com/package/protractor-video-reporter/tutorial
• https://github.com/tepez/protractor-video-reporter/issues/5
• https://github.com/tepez/protractor-video-reporter

Unfortunately for me - due to some other infrastructure constraints - this got de-prioritized. But hopefully I will be getting back to this soon.

Slides from vodQA Pune - Less Talk, Only Action! now available

vodQA-Pune - Less Talk, Only Action! was held on on Saturday, 27th Aug 2016, 8.30am - 5.30pm at ThoughtWorks, Pune.

Agenda

Abstracts with Slides

1. Automating Web Analytics - Why? How?

Do you know –

• What is Web Analytics?How does Web Analytics work?
• Why is it important? How to test Web Analytics?
• How can we ensure correct data is sent to the Web Analytics server, every time, for all the actions?

Attend this workshop to learn ‘What is Web Analytics?’ and why it is an extremely important aspect of Software Development & Testing for your product / service to succeed!

We will share some techniques for testing Web Analytics - in a non-automated way - and why that is very challenging and error-prone.

We will learn, via hands-on activity, about WAAT - Web Analytics Automation Testing Framework (https://essenceoftesting.blogspot.com/search/label/waat) - an open-source solution, to automate validation of correct information / tags being sent to the Web Analytic server for different user actions as part of your regular Selenium-WebDriver Test Automation Framework.

Lastly, we will see how the impact of Analytics has changed dramatically with more adoption and spread of IoT (Internet of Things) and Big Data, and what we need to do to be part of the change, if not influencers of change!

Slides: automating-web-analytics

2. Performance testing with Gatling for Beginners

Gatling is a server side performance testing tool. This workshop aims at giving introduction to Gatling and facilitating participants to write their first performance tests using Gatling.

• Brief intro to Gatling
• Using Postman to check stub server(created using mountebank for workshop)
• Write a sample test using gatling (pre set up machines are provided)

Slides: gatling-performance-workshop

3. Game of Test Automation

We are going to use a game to work out the WHY, WHAT and HOW of test automation within the context of consumers, application, skillset, mindset, etc.

4. Security Testing - Operation Vijay

It is now the days of the web. All businesses move their applications online for their customers to use.

Many of these applications contain critical data of the customers such as credit card details, their personal details and so on. These data are very valuable. If these data fall in the wrong hands, they can have disastrous consequences.

Attacks on these systems can destroy the trust that the customers have for the business, they can cause great losses to the customers as well as the business, and so on.

The motivation behind attacks could be different like earning money, earning popularity, destroying a competitor company, etc.

No matter what the intention of the attack is, we need to develop safe applications and we need to know the various vulnerabilities and the consequences of our decisions when we develop applications.

Similarly, we should be aware of the various vulnerabilities before we test the applications so that we can try and exploit it during the testing phase and ensure better quality and safer applications.

Slides: security-testing-operation-vijay

5. Automate your Mobile tests with Appium

• Introduction to Appium.
• Appium design for Android and iOS
• How to locate elements on android and iOS applications(Inspector).
• Hands-on code snippet for android and iOS(Wordpress as sample app)
• Generating reports (ExtentReports)
• Evolve code snippet written above into a Page Object Framework.
  

Slides: mobile-automation-using-appium

6. Increase Automation to REST

• What are web services and why we use them?
• How to test a web service in multiple ways?
• Increased familiarity with automation

Tools used : rest client, postman (mention alternatives), unirest JAVA and requests python

Slides: increase-automation-to-rest, api-webservice-setup-instructions

7. Let's cook Cucumber

In this workshop we will be covering:

• Advantages of BDD through cucumber example.
• Framework setup along with JAVA and Selenium.
• Writing one end to end test case in real world.
• If time permits - will be covering basic Refactoring
  

Slides: lets-cook-cucumber
 

8. Mobile Automation using Espresso

Imagine a situation where every commit spits out a build that can be deployed to production with confidence. In today's startup era, this can be a huge boost to business as it will reduce the time to market. UI Automation for mobile apps, be it native or hybrid, has been painful since long. But with mature frameworks coming up and Google/Apple realizing the importance of such tools, UI Automation is gaining traction in the mobile space. 

This talk is basically to understand what and why of espresso along with automating a simple scenario using espresso.

Slides: getting-high-on-espresso

Agile Testing & Patterns in Test Automation Framework coming to Singapore

I will be doing a 1-day workshop on Agile Testing and also a talk on "Patterns of a Good Test Automation Framework" as part of Unicom's World Conference Next Gen Testing Summit on Oct 13-14 2016 in Singapore.

See the site for more details, or contact me for more information on the same.

WAAT 2.0 (BETA) available for use

Very excited to share that I, along with a few collaborators of WAAT did a workshop in vodQA, Pune on Sat, 27th Aug about Automating Web Analytics - Why? How?. 
We implemented the automation using WAAT.

Also, am very excited to announce that WAAT v2.0 BETA for Java is now available from the WAAT project.

The slides from the workshop are available here

Automating Web Analytics from Anand Bagmar

Here are some details of the workshop:

Abstract

Do you know –

• What is Web Analytics?

• How does Web Analytics work?

• Why is it important? How to test Web Analytics?

• How can we ensure correct data is sent to the Web Analytics server, every time, for all the actions?

Attend this workshop to learn ‘What is Web Analytics?’ and why it is an extremely important aspect of Software Development & Testing for your product / service to succeed!

 
We will share some techniques for testing Web Analytics - in a non-automated way - and why that is very challenging and error-prone.

 
We will learn, via hands-on activity, about WAAT - Web Analytics Automation Testing Framework (https://essenceoftesting.blogspot.com/search/label/waat) - an open-source solution, to automate validation of correct information / tags being sent to the Web Analytic server for different user actions as part of your regular Selenium-WebDriver Test Automation Framework.

 
Lastly, we will see how the impact of Analytics has changed dramatically with more adoption and spread of IoT (Internet of Things) and Big Data, and what we need to do to be part of the change, if not influencers of change!

Pre-Reqs

• JDK 1.8
• IDE (Eclipse / IntelliJ Idea / etc.)
• Git (and add it to path)
• Gradle 2.3 (and add it to path)
• Firefox browser compatible with Selenium v2.53.1 (v46)
• Also refer here for steps to run Appium tests using WAAT 

 

Sample Code

- Create a directory WAAT-Workshop and run
git clone git@github.com:anandbagmar/waat-sample-java.git
- If Using any existing framework: download WAAT-all-v2.0_BETA.jar
http://goo.gl/8bfyaJ
- To run sample code - from waat-sample-java directory - run the command -
gradle clean build 

Please reach out if you need help in using WAAT!

vodQA Pune - Less Talk, Only Action - Agenda, Abstract and workshop pre-requisites

vodQA-Pune - Less Talk, Only Action!

When? - Saturday, 27th Aug 2016, 8.30am - 5.30pm

Where? 

ThoughtWorks, Pune 
6th Floor, Binarius Building, 
Beside Sales Tax Office, Shastrinagar, Yerawada 
Pune, Maharashtra 411006 

Map: http://goo.gl/maps/KfuJG

What?

vodQA Pune - Less Talk, Only Action! 
- Registration - 8.30am- 9.15am
- Welcome note - 9.15am

Agenda

Abstracts

1. Automating Web Analytics - Why? How?

Do you know –

• What is Web Analytics?

• How does Web Analytics work?

• Why is it important? How to test Web Analytics?

• How can we ensure correct data is sent to the Web Analytics server, every time, for all the actions?

Attend this workshop to learn ‘What is Web Analytics?’ and why it is an extremely important aspect of Software Development & Testing for your product / service to succeed!

We will share some techniques for testing Web Analytics - in a non-automated way - and why that is very challenging and error-prone.

We will learn, via hands-on activity, about WAAT - Web Analytics Automation Testing Framework (https://essenceoftesting.blogspot.com/search/label/waat) - an open-source solution, to automate validation of correct information / tags being sent to the Web Analytic server for different user actions as part of your regular Selenium-WebDriver Test Automation Framework.

Lastly, we will see how the impact of Analytics has changed dramatically with more adoption and spread of IoT (Internet of Things) and Big Data, and what we need to do to be part of the change, if not influencers of change!

Pre-Reqs:

• JDK 1.8
• IDE (Eclipse / IntelliJ Idea / etc.)
• Git
• Firefox browser compatible with Selenium v2.53.1 (v46)
• Also refer here for steps to run Appium tests using WAAT

2. Performance testing with Gatling for Beginners

Gatling is a server side performance testing tool. This workshop aims at giving introduction to Gatling and facilitating participants to write their first performance tests using Gatling.

> Brief intro to Gatling

> Using Postman to check stub server(created using mountebank for workshop)

> Write a sample test using gatling (pre set up machines are provided)

Pre-Reqs:

• Participants should know at least one programming language well (Gatling tests are written in Scala. But participants need not know Scala)
• Prior knowledge of what 'Performance testing is?' would help.

System requirements:

• Homebrew (Package Manager for Mac)
• Java 8
• Gradle
• Intellij IDE(Add Scala plug in)

3. Game of Test Automation

We are going to use a game to work out the WHY, WHAT and HOW of test automation within the context of consumers, application, skillset, mindset, etc.

Pre-Reqs: None

4. Security Testing - Operation Vijay

It is now the days of the web. All businesses move their applications online for their customers to use.

Many of these applications contain critical data of the customers such as credit card details, their personal details and so on. These data are very valuable. If these data fall in the wrong hands, they can have disastrous consequences.

Attacks on these systems can destroy the trust that the customers have for the business, they can cause great losses to the customers as well as the business, and so on.

The motivation behind attacks could be different like earning money, earning popularity, destroying a competitor company, etc.

No matter what the intention of the attack is, we need to develop safe applications and we need to know the various vulnerabilities and the consequences of our decisions when we develop applications.

Similarly, we should be aware of the various vulnerabilities before we test the applications so that we can try and exploit it during the testing phase and ensure better quality and safer applications.

Pre-Reqs:

• Java
• Modern Browser like Firefox/Chrome

5. Automate your Mobile tests with Appium

Introduction to Appium.

Appium design for Android and iOS

How to locate elements on android and iOS applications(Inspector).

Hands-on code snippet for android and iOS(Wordpress as sample app)

Generating reports (ExtentReports)

Evolve code snippet written above into a Page Object Framework.

Pre-Reqs:

• IntelliJ
• Java 1.8
• MAVEN
• Node
• Adb - android debug bridge
• Appium installed through node(npm install -g appium). Once the installation goes well. Hit command “appium” without quote and you should see Welcome message from appium.
• Download & install Virtual Box from here
• Download the basic version on Genymotion Android Emulator from here
• Make sure you chose the basic version which is free for use(Go to individual tab).
• You will be asked to create account with Genymotion before downloading the Genymotion package/installer
• Confirm your account by pressing the link sent to your registered email id
• Once you download, install Genymotion Android Emulator by following the instructions here
• Once you install, open/start Genymotion and click on Add(+) link to add a new virtual device and press Signin and login
• Select Android Version 6.0.0 and Device model Google Nexus 5 or 6. For more details follow the instruction mentioned here
• Select from available option, Press Next and wait till download finishes.
• Press start button & you should see emulator running.

6. Increase Automation to REST

What are web services and why we use them?

How to test a web service in multiple ways?

Increased familiarity with automation

Tools used : rest client, postman (mention alternatives), unirest JAVA and requests python

Pre-Reqs:

• IntelliJ
• MAVEN
• CHROME and POSTMAN

7. Let's cook Cucumber

In this workshop we will be covering:

• Advantages of BDD through cucumber example.
• Framework setup along with JAVA and Selenium.
• Writing one end to end test case in real world.
• If time permits - will be covering basic Refactoring

Pre-Reqs:

• IntelliJ
• Java 1.8
• Maven
• Firefox 46

8. Mobile Automation using Espresso

Pre-Reqs:

• Ensure you have Java 7 or beyond, installed on your box. If not you can download it from here 
• Download & install Virtual Box from here
• Download and install Android Studio from here. 
• Also kindly download the android command line tools from the above link.
• If you open android studio, it will ask you download & setup Android API and build tools which is useful for debug & compile our apps so please follow the steps. Ensure you have following things installed before you come for workshop and downloading it takes lot of time. 
  • Android SDK build-tools v23.0
  • Android API 5.0 or above
  • Install Android Support library from Extras. 
  • Please google in case of any issues while installing SDK
• Download the basic version on Genymotion Android Emulator from here
  • Make sure you chose the basic version which is free for use(Go to individual tab).
  • You will be asked to create account with Genymotion before downloading the Genymotion package/installer
  • Confirm your account by pressing the link sent to your registered email id
  • Once you download, install Genymotion Android Emulator by following the instructions here
  • Once you install, open/start Genymotion and click on Add(+) link to add a new virtual device and press Signin and login 
  • Select Android Version 6.0.0 and Device model Google Nexus 5 or 6. For more details follow the instruction mentioned here
  • Select from available option, Press Next and wait till download finishes.
  • Press start button & you should see emulator running.
  • You’re all set for the event.