How to handle Select certificate popup from Chrome using Selenium WebDriver?

For one of the applications, when I run some Selenium WebDriver tests. I see a Select a certificate popup. I am not able to handle this, and hence my test fails. 

Has anyone been able to handle this popup, or rather, avoid this popup being shown?

If I manually select a certificate and click OK, I am prompted for my mac credentials:

Thanks in advance!

Ways to implement fakesms service / otp validation in Functional Test Automation

I had asked a question in my earlier blog post about 

How do you automate OTP related test scenarios? Do you use a fake SMS service? Does it have restapi to query the SMS messages? geography support? 

A lot of people replied to it directly on the post, or via twitter and LinkedIn. I thank them all for their responses and suggestions!

 

Here is the summary of the suggestions received:

• https://twitter.com/BagmarAnand/status/1483697448375701505
• https://www.proovl.com/test
• https://freephonenum.com/
• https://www.bulksms.com/test/
• https://smstome.com/
• https://oksms.org/
• https://simpletexting.com/virtual-phone-numbers/
• In addition, there are paid services that will allow you to integrate this as part of functional automation - https://www.linkedin.com/posts/anandbagmar_otp-test-scenarios-activity-6889461246991613952-KbrC/ - lot of discussion here 

In addition, if anyone needs to test email flows, getnada.com is a great way you can automate and validate emails. They have rest apis for the same.

 

These solutions will allow you to automate seemingly difficult to automate functionality, WITHOUT having to create "testing-hooks" in your product. Hence, you will be able to test the code that will be shipped to Production, which is very important!

Hope this helps others who may be looking for similar things.

Why not to use PageFactory and FindsBy in Selenium WebDriver

Many users of Selenium WebDriver may be using the PageFactory created by Simon Stewart. However, it is not a good idea to use it.

You may be thinking why should I not use it? It is so easy to use it, and its popular.

Well, here are 2 reasons why you should not use the PageFactory:

Reason #1. Simon Stewart (https://twitter.com/shs96c), the creator of WebDriver, and the PageFactory himself says, do not use it. It is not recommended.

The `FindsBy` annotation isn't recommended, because the PageFactory class is really badly implemented and inflexible, but it's not going away in the java bindings.

The `FindsByX` interfaces are going away. Better to use a `By` locator and use that.

 

https://twitter.com/shs96c/status/1196865907185868801

Reason #2: While Reason #1 should have been sufficient, many people implementing automation using Selenium WebDriver do not know, or did not pay heed to what Simon said. So another WebDriver & WATIR contributor, Titus Fortner (https://twitter.com/titusfortner) explained in detail why using PageFactory is not a good idea in his blog post - https://titusfortner.com/2021/02/03/page-factory-optimization.html. 

 

I sincerely hope these reasons are sufficient for you to move away from the PageFactory and use something more efficient. 

 

Using fakesms service in Functional Test Automation

How do you automate OTP related test scenarios? Do you use a fake SMS service? Does it have restapi to query the SMS messages? geography support? 

To clarify - this needs to be done as part of my functional test automation, where,

• the test could be running against a browser, where the browser does not have access to the phone, or,
• the test could be running against a real mobile device (without SIM), so no way to receive the SMS, or,
  
• the test could be running against an emulator (no SIM), so no way to receive the SMS

Scenarios include: login, payment, SMS content 

Hence I am thinking about using a fakesms service which has API access capabilities to retrieve the SMS. This will help when running automation on browser or devices / emulators without SIM.

Note:

• There is no access to DB or API to query the OTP. 
• I don't mind using a paid service

 

Thanks in advance for your help!

Test Automation Pyramid

 

Business-Layer Page-Object Pattern

Business-Layer Page-Object Pattern for Functional / System / end-2-end Test Automation

1. Tests should talk business language
2. The test is deterministic, for a specific scenario. The test implementation is an orchestration of corresponding business operations
3. Business layer is an abstraction layer between the test intent & page objects
4. Implementation of business layer method is essentially an orchestration of other business operations, or for the granular business operation, an orchestration of page objects
5. The business layer method does the assertions of expectations
6. There should be no assertions in page objects
7. Each operation (in business or page object) being successful means there are a defined number of methods / operations the product can now do (as you are driving the product under test to do your bidding)
8. Hence, to #7, each operation can have one 1 valid page / business object as its return type

See the sample tests implemented in teswiz for an example of Business-Layer Page-Object pattern implementation.

Automating Functional / End-2-End Tests Across Multiple Platforms

Do you want to know more about an approach to Functional (end-to-end) Test Automation that works for a product available on multiple platforms?

If yes, then read my blog post on Automating Functional / End-2-End Tests Across Multiple Platforms which shares details on the thought process & criteria involved in creating a solution that includes how to write the tests, and run it across the multiple platforms without any code change.

Lastly, the open-sourced solution - teswiz also has examples on how to implement a test that orchestrates the simulation between multiple devices / browsers to simulate multiple users interacting with each other as part of the same test.

Visual Assertions - not another buzzword

Visual Testing and Visual Assertions may seem like yet another buzzword in the Software industry.

Being curious, I did an experiment using Applitools Visual AI to see if this is something that can genuinely help, or if it is a buzzword. You can read about this experiment, refer to the code and see the resulting data from this post - "Visual Assertions - Hype or Reality?".

Getting started with implementing Automation

Getting started with implementing tests for automation (web or native apps) may seem daunting for those who are doing this for the first time. 

Assuming you are using open-source tooling like Selenium or Appium, there are multiple ways you can get started.

1. DIY - Build your own framework by scripting based on the documentation

2. Use Selenium-IDE for quick record and playback

3. Use TestProject Recorder for quick record and playback

4. Use TestProject SDK to build your own custom scripts for automating the tests
   
   

Each of the above approaches has its own pros-and-cons. Let's look at this in some detail:

Approach #1 - DIY - Build your own framework

Selenium: https://www.selenium.dev/documentation/en/

Appium: https://appium.io/docs/en/about-appium/intro/

Pros	Cons
You can build all features and capabilities as per your design & requirement	*You need to learn a programming language
	*You have to build everyone on your own (though you can use supporting libraries)

* Depending on the context of the team, the above points can also be considered as an advantage

Approach #2 - Selenium-IDE

https://www.selenium.dev/selenium-ide

Pros	Cons
Easy to set up	Basic reports
Works in Chrome & Firefox	Works only for automating Web applications
Code can be exported in various formats
Recorded tests can be run from command line
Tests can be run in your own CI
Will always be in-sync with underlying WebDriver

Approach #3 - TestProject Recorder

https://testproject.io/easy-test-automation/

Pros	Cons
Advanced recorder (lot of actions, validations, self-healing, customisations possible, and a lot of community Addons)	Recorder works only in Chrome, but tests can be executed on all browsers
Recorder works for Web applications as well as Native Apps (in real devices or emulators) for Android and iOS (even iOS on Windows machine)	Generated code is very simple - good as a reference to see how the underlying implementation / interaction is done
TestProject agent automatically determines all available browsers available and devices connected to the machine and execution can be customised accordingly	Each recorded test needs to be exported individually. No concept of reuse in this approach
Can schedule test runs as one-time, or repeated activity via build-in scheduler / CI/CD tool integrations or via their RESTful API
Reports are comprehensive with meaningful data, including screenshots and option to download to PDF format
Code can be generated from the recorded script
Can share tests easily using the "Share test" feature

Approach #4 - TestProject SDK

https://testproject.io/advanced-scripting-capabilities

Pros	Cons
Probably the most powerful way of these 4 approaches as it uses WebDriver / Appium under the hood. Get the power of building your own framework, while reusing out-of-the-box features like driver management, automatic reporting, etc.	You need to learn a programming language
Driver management is TestProject responsibility. Test implementer can focus on automating tests

Questions at at Taquelah - Does your functional automation really add value?

I spoke at Taquelah Lightning Talks on one of my favorite topics - 

Does your functional automation really add value?

 

You can find the slides here  - https://www.slideshare.net/abagmar/does-your-functional-automation-really-add-value

Some references:

https://essenceoftesting.blogspot.com/2020/07/does-your-functional-automation-really.html

https://essenceoftesting.blogspot.com/2020/03/tracking-functional-coverage.html

Waiting for parallel features to complete in Karate / IntelliJ

Anyone using KarateDSL (https://github.com/intuit/karate) getting the below mentioned error?

[main] INFO http://com.intuit.karate.Runner - waiting for parallel features to complete ...

Restarting IntelliJ did not help.

Karate version: 0.9.5

JDK: Adopt Open JDK 11.0.8

Any tips / idea how to resolve it?

Overcoming chromedriver version compatibility issues the right way

I encountered an interesting challenge recently when doing Native Android / iOS app automation - this was related to Chrome browser versions getting updated automatically and my tests failing because of errors like:

org.openqa.selenium.SessionNotCreatedException: session not created: This version of ChromeDriver only supports Chrome version 74 23:04:25 (Driver info: chromedriver=74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729@{#29}),platform=Windows NT 6.3.9600 x86_64) (WARNING: The server did not provide any stacktrace information)

So I asked a question on LinkedIn


And I tweeted asking how to manage ChromeDriver version when running WebDriver / Appium tests.

I have a #appium / #java #framework.
What is a good way to keep #chromedriver updated with version of #chrome via #scripts? I do not want to do this manually - since I have 3+ #jenkins agents (#linux + #mac mini) and 7+ team members who need to keep doing this #osx #command

— Anand Bagmar (@BagmarAnand) September 20, 2019

The answer was common and obvious – use WebDriverManager. This is a beautiful, simple and indeed the right answer and solution to the problem.

However, that was a partial answer for me. 

Here is my context and problem statement in detail:

• My Test Automation Framework is based on Java / Appium and I use AppiumTestDistribution (ATD) 
• ATD is open-source, and takes away my pain and effort of managing appium and the devices and also takes care of running the tests in parallel or distributed mode, on android as well as iOS
• In my local lab setup, I have many different android devices connected - which run tests as directed by ATD
• Since you cannot control how Google PlayStore / Apple App Store pushes out new versions of apps for different android / iOS versions on devices, it is easily possible to end up with different versions of chrome browser in your device lab. When this happens, the tests start failing because of chromedriver incompatibility issues.

Once I was very kindly reminded by the community about WebDriverManager (which I had forgotten about), I now knew what was to be done.

I looked at the ATD code, and realised that it was using the default chromedriver version as setup when I had installed appium. This chromedriver was being used when instantiating a new instance of the AndroidDriver.

So I submitted a PR for ATD - which essentially did the following:

• Query the chrome browser versions on each connected device
• For the **highest version of the browser, use WebDriverManager and get the appropriate chromedriver downloaded
• Pass the path to the correct chromedriver when creating an instance of the AndroidDriver

**highest version - what does that mean? Well, I also got confused initially. But the answer was simple. On some devices, the Chrome browser is installed by default, as a system app. This cannot be removed. So as new versions of the browser get installed, the default Chrome system app is always there. So when you query for the versions of Chrome on the device, you will see 2 such versions. My code logic was to get all these versions, and pick the highest version from them.

Here is the code snippet of how I solved the problem:

Special thanks to Sai Krishna for quickly approving and merging this PR.

Hope this provides more information about my problem statement, and how I used your suggestion for WebDriverManager to solve the problem.

Analytics - The Brain of the Software

An Analogy 

I am not a doctor, nor did I enjoy biology too much in my curriculum as a student.  However, I do know that the body has many organs and each organ plays a vital role in the well being of the individual.

Each organ has to:

• function correctly (movement, senses, core functions, etc.)

• has to perform as per expectations in different conditions the individual may be going through (walking, running, swimming, etc.)

• has to be secure from external parameters (heat, cold, rain, what we eat / drink, etc.)

• has to have a proper user experience (ex: if the human hands had webs like ducks, would we be able to hold a pen correctly to write?

• I would like to think of the brain as the super computer which keeps track of what is going on in the body, if each piece playing its part correctly, or not. And if there is something unexpected going on, then there are mechanisms of giving that feedback internally and externally so that course correction would be possible.

How does this relate to software?

Software is similar in some ways. For any software product to work, the following needs to be done:

Functionality works as expected

• The architecture, testability of the system will allow for various types of testing activities to be performed on the software to ensure everything works as expected
• Test Automation practices will give you quick feedback

There is a plethora of open-source and commercial tools in this space to help in this regard - the most popular open-source tools being Selenium and Appium.

Software is performant

• We can do performance testing at various different levels to ensure at different loads and conditions, the users will be able to use the product in a seamless fashion
• There are many tools to assist in Performance Testing - some popular ones being JMeter and Gatling.

Software is secure

• Building and testing for security is critical as you do not want user information to be leaked or manipulated and neither do you want to allow external forces to control / manipulate your product behaviour and control
• The Test Automation Pyramid hence also includes NFRs

User experience is validated, and consistent

• In the age of CD (Continuous Delivery & Continuous Deployment), you need to ensure your user experience across all your software delivery means (browsers, mobile-browsers, native apps for mobiles and tablets, etc.) is consistent and users do not face the brunt of UI / look-and-feel issues in the software at the cost of new features
• This is a relatively new domain - but there are already many tools to help in this spaces as well - the most popular one (in terms of integration, usage and features) being the AI-powered Applitools

Visual Validation is the new tip of the Test Automation Pyramid!

What is the brain of the software?

The above is all good, and known in various ways. But what is the "brain" of the software? How does one know if everything is working fine or not? Who will receive the feedback and how do we take corrective action on this?

Analytics is that piece in the Software product that functions as the brain. It keeps collecting data about each important piece of software, and provides feedback on the same.

I have come across some extreme examples of Business / Organizations who have all their eggs in one basket - in terms of

• understand their Consumers (engagement / usage / patterns / etc.),
• understand usage of product features, and,
• do all revenue-related book-keeping

This is all done purely on Analytics! Hence, to say “Business runs on Analytics, and it may be OK for some product / user features to not work correctly, but Analytics should always work” - is not a myth!

What this means is Analytics is more important now, than before.

Unfortunately, Analytics is not known much to the Software Dev + Test community. We know it very superficially - and do what is required to implement it and quickly test it out. But what is analytics? Why is it important? What is the impact of this not working well? Not many think about this.

I have been testing Analytics since 2010 ... and the kind of insights I have been able to get about the product have been huge! I have been able to contribute back to the team and help build better quality software as a result.

But I have to be honest - it is painful to test Analytics. And that is why I created an open-source framework - WAAT - to help automate some of this testing activities.

I also do workshops to help people learn more about Analytics, its importance, and how they can automate this as well.

In the workshop, I do not assume anything and approach is to discuss and learn by example and practice, the following

• How does Analytics works (for Web and Mobile)?
• Test Analytics manually in different ways
• Test Analytics via the final reports
• Why some Automation strategies will work, and some WILL NOT WORK (based on my experience)!
• We will see a demo of the Automation running for the same.
• Time permitting, we will setup running some Automation scripts on your machine to validate the same

Takeaways from the workshop

We will learn by practicing the following:

• What is Analytics?
• Techniques to test analytics manually.
• How to automate the validation of analytics, via a demo, and if time permits, run the automation from your machine as well.

Hope this post helps you understand the importance of Analytics and why you need to know more about it. Do reach out to me if you want to learn more about it.

Next upcoming Analytics workshop is in TestBash Australia 2019. Let me know if you would be interested in attending the same