Why not to use PageFactory and FindsBy in Selenium WebDriver

Many users of Selenium WebDriver may be using the PageFactory created by Simon Stewart. However, it is not a good idea to use it.

You may be thinking why should I not use it? It is so easy to use it, and its popular.

Well, here are 2 reasons why you should not use the PageFactory:

Reason #1. Simon Stewart (https://twitter.com/shs96c), the creator of WebDriver, and the PageFactory himself says, do not use it. It is not recommended.

The `FindsBy` annotation isn't recommended, because the PageFactory class is really badly implemented and inflexible, but it's not going away in the java bindings.

The `FindsByX` interfaces are going away. Better to use a `By` locator and use that.

 

https://twitter.com/shs96c/status/1196865907185868801

Reason #2: While Reason #1 should have been sufficient, many people implementing automation using Selenium WebDriver do not know, or did not pay heed to what Simon said. So another WebDriver & WATIR contributor, Titus Fortner (https://twitter.com/titusfortner) explained in detail why using PageFactory is not a good idea in his blog post - https://titusfortner.com/2021/02/03/page-factory-optimization.html. 

 

I sincerely hope these reasons are sufficient for you to move away from the PageFactory and use something more efficient. 

 

Business-Layer Page-Object Pattern

Business-Layer Page-Object Pattern for Functional / System / end-2-end Test Automation

1. Tests should talk business language
2. The test is deterministic, for a specific scenario. The test implementation is an orchestration of corresponding business operations
3. Business layer is an abstraction layer between the test intent & page objects
4. Implementation of business layer method is essentially an orchestration of other business operations, or for the granular business operation, an orchestration of page objects
5. The business layer method does the assertions of expectations
6. There should be no assertions in page objects
7. Each operation (in business or page object) being successful means there are a defined number of methods / operations the product can now do (as you are driving the product under test to do your bidding)
8. Hence, to #7, each operation can have one 1 valid page / business object as its return type

See the sample tests implemented in teswiz for an example of Business-Layer Page-Object pattern implementation.

Long time no see? Where have I been

It has been a few months since I published anything on my blog. Does not mean I have not been learning or experimenting with new ideas. In fact, in the past few months I have been privileged to have my articles published on Applitools and Test Project blogs.

Below is the link to all those articles, for which I have received very kind reviews and comments on LinkedIn and Twitter.

Apart from this, I have also been contributing to open source - namely - Selenium, AppiumTestDistribution and building an open-source kickstarter project for API testing using Karate and for end-2-end testing for Android, iOS, Windows, Mac & Web.

Lastly, I have also been speaking in virtual conferences, webinars and last week I also recorded a podcast, which will be available soon.

The end of Smoke, Sanity and Regression

https://applitools.com/blog/end-smoke-sanity-regression/

Do we need Smoke, Sanity, Regression suites?

• Do not blindly start with classifying your tests in different categories. Challenge yourself to do better!
• Have a Test Automation strategy and know your test automation framework objective & criteria (“Test Automation in the World of AI & ML” highlights various criteria to be considered to build a good Test Automation Framework)
• Choose the toolset wisely
• After all the correct (subjective) approaches taken, if your test execution (in a single browser) is still taking more than say, 10 min for execution, then you can run your tests in parallel, and subsequently, split the test suite into smaller suites which can give you progressive indication of quality
• Applitools with its AI-power algorithms can make your functional tests lean, simple, robust and includes UI / UX validation
• Applitools Ultrafast Grid will remove the need for Cross-Browser testing, and instead with a single test execution run, validate functionality & UI / Visual rendering for all supported Browsers & Viewports

Design Patterns in Test Automation

https://blog.testproject.io/2020/06/29/design-patterns-in-test-automation/
 

Criteria for building a Test Automation Framework

Writing code is easy, but writing good code is not as easy. Here are the reasons why I say this:

• “Good” is subjective.
• “Good” depends on the context & overall objective.

Similarly, implementing automated test cases is easy (as seen from the getting started example shared earlier). However, scaling this up to be able to implement and run a huge number of tests quickly and efficiently, against an evolving product is not easy!

I refer to a few principles when building a Test Automation Framework. They are:

• Based on the context & (current + upcoming) functionality of your product-under-test, define the overall objective of Automation Testing.
• Based on the objective defined above, determine the criteria and requirements from your Test Automation Framework. Refer to my post on “Test Automation in the World of AI & ML” for details on various aspects you need to consider to build a robust Test Automation Framework. Also, you might find these articles interesting to learn how to select the best tool for your requirements:
• Criteria for Selecting the Right Functional Testing Tools
• How to Select the Best Tool – Research Process
• How To Select The Right Test Automation Tool

Stop the Retries in Tests & Reruns of Failing Tests

https://applitools.com/blog/uncover-flaky-tests/
 

Takeaways

• Recognise reasons why tests could be flaky / intermittent
• Critique band-aid approach to fixing flakiness in tests
• Discuss techniques to identify reasons for test flakiness
• Fix the root-cause, not the symptoms to make your tests stable, robust and scalable!

Measuring Code Coverage from API Workflow & Functional UI Tests

https://blog.testproject.io/2020/11/24/measuring-code-coverage-from-api-workflow-functional-ui-tests/

 
Why is the Functional Coverage important?

I choose an approach keeping the 80-20 rule in mind. The information the report provides should be sufficient to understand the current state, and take decisions on “what’s next”. For areas that need additional clarity, I can then talk with the team, explore the code to get to the next level of details. This makes it a very collaborative way of working, and joint-ownership of quality! 🚀

You can choose your own way to implement Functional Coverage – based on your context of team, skills, capability, tech-stack, etc.

Test Automation in the World of AI and ML

My article on "Test Automation in the World of AI & ML" recently got published on InfoQ.


Here are the key takeaways mentioned in the article -

• There are many criteria to be considered before building framework / selecting tools for Functional Test Automation
• It is very important to prioritise framework / tools capabilities needed for the software-under-test
• A good, scalable Test Automation Framework that provides fast and reliable feedback to the team enables collaboration and CI/CD
• Debugging / RCA (root cause analysis) and support for libraries / tools used is an afterthought in most cases. Do not fall in that trap.
• There are some promising commercial tools that fit seamlessly in the Agile way of working. Depending on the complete context, these tools may be a good choice over building your own framework for Functional Automation.

You can read the full article from here. 

Looking forward to comments on the same!

.

Agile Testing & Patterns for a good Test Automation Frameworks

2018 started with a bang! I got an opportunity to speak and share my experiences in 2 rocking meetups.

Patterns of a "good" Test Automation Framework

TechnoWise meetup on 13th Jan on Patterns of a "good" Test Automation Framework went very well, with lot of interaction and discussions along the way.

Patterns of a “good” test automation framework from Anand Bagmar

What is Agile Testing? How does Automation Help?

Then, there was an impromptu meetup setup by a very proactive ISQA community at GO-JEK office in Jakarta, Indonesia on 16th Jan. The topic there was What is Agile Testing? How does Automation help? In this meetup, I also covered aspects of Career Path of a Tester, and QA Skills and Capabilities.

There were many amazing experiences from this meetup -

• The ISQA community is very active. The meetup was setup in literally a few days and there were over 150+ attendees
• The GO-JEK office space is a very fun place. They actually have an auditorium in the office to host meetups and similar activities, apparently, once a week!!
• The questions / interactions with the attendees were very insightful

Here are slides shared in the meetup. I will share the link of the video as well once available.

What is Agile Testing? How does Automation help? from Anand Bagmar

Patterns in Test Automation Framework at STPCon

I spoke about Patterns of a "good" Test Automation Framework at STPCon 2017. Here are the details from the talk.

Abstract

Building a Test Automation Framework is easy – there are so many resources / guides / blogs / etc. available to help you get started and help solve the issues you get along the journey.

However, building a “good” Test Automation Framework is not very easy. There are a lot of principles and practices you need to use, in the right context, with a good set of skills required to make the Test Automation Framework maintainable, scalable and reusable.

Design Patterns play a big role in helping achieve this goal of building a good and robust framework.

In this talk, we will talk about, and see examples of various types of patterns you can use for:

• Build your Test Automation Framework
• Test Data Management

Using these patterns you will be able to build a good framework, that will help keep your tests running fast, and reliably in your CI / CD setup!

Session Takeaways:

• Patterns for building Test Automation Framework.
• Patterns for Test Data Management, with pros and cons of each.

Slides

Patterns of a “good” test automation framework from Anand Bagmar

Pictures

Agile Testing & Patterns in Test Automation Framework coming to Singapore

I will be doing a 1-day workshop on Agile Testing and also a talk on "Patterns of a Good Test Automation Framework" as part of Unicom's World Conference Next Gen Testing Summit on Oct 13-14 2016 in Singapore.

See the site for more details, or contact me for more information on the same.

Slides & Video from Test Data - Food for Test Automation Framework

As posted earlier, I spoke about "Test Data - Food for your Test Automation Framework!" at Selenium Conference 2016 in Bangalore on 24th June 2016 in front of a packed and a very interactive audience. What a great time it was!

The video for that session is finally available! I have updated the original post as well as linked the slides & video with this post as well.

Video

Slides

Test Data - Food for your Test Automation Framework from Anand Bagmar

The recurring pattern of Patterns

On popular demand, I spoke in TechJam, the 2nd time around on Tuesday, 21st June on "Patterns of a 'good' Test Automation Framework, Locators & Data".

This time, the TechJam team at IDeaS used a different setup - a new conference room, with live-streaming via Google Hangout. It was indeed a great setup - with around 40 people able to sit and participate in discussions for over an hour - about Patterns in Test Automation.

Slides

Patterns in Test Automation from Anand Bagmar

Video

Pictures

Test Data - Food for Test Automation Framework at Selenium Conference 2016

[Updated - Pictures added, Video added]

I spoke about "Test Data - Food for your Test Automation Framework!" at Selenium Conference 2016 in Bangalore on 24th July 2016 in front of a packed and a very interactive audience. What a great time it was!

Here is some information about the session - 

Abstract

Building a Test Automation Framework is easy - there are so many resources / guides / blogs / etc. available to help you get started and help solve the issues you get along the journey.

Teams already building 1000s of tests of various types - UI, web service-based, integration, unit, etc. is a proof of that.

However, building a "good" Test Automation Framework is not very easy. There are a lot of principles and practices you need to use, in the right context, with a good set of skills required to make the Test Automation Framework maintainable, scalable and reusable.

In this talk, we will focus on one of the critical aspects and patterns in building the Test Automation framework - Test Data!

We will look at different data patterns as options and techniques how to create, manage, use, reuse Test Data in a way to keep the tests running in an reliable and deterministic way. We will also discuss what questions to ask, what things to think about in selecting your approach for Test Data!

This discussion will be applicable for any type of Test Automation (web / mobile / desktop), but, we will focus primarily on UI automation frameworks, ex. using Selenium.

Slides

Test Data - Food for your Test Automation Framework from Anand Bagmar

Video

Pictures

Patterns in Test Automation presented at Agile India 2016

I spoke about Patterns of a “good” Test Automation Framework, Locators & Data! in Agile India 2016 in front of a packed room of Developers, Testers, BAs and POs.

Below is the abstract, slides & video of the talk.

Patterns of a “good” Test Automation Framework, Locators & Data!

Building a Test Automation Framework is easy - there are so many resources / guides / blogs / etc. available to help you get started and help solve the issues you get along the journey.

However, building a "good" Test Automation Framework is not very easy. There are a lot of principles and practices you need to use, in the right context, with a good set of skills required to make the Test Automation Framework maintainable, scalable and reusable.

Design Patterns play a big role in helping achieve this goal of building a good and robust framework. 

In this talk, we will talk about, and see examples of various types of patterns you can use for:

1. Build your Test Automation Framework
2. Test Data Management
3. Locators / IDs (for finding / interacting with elements in the browser / app)

Learning Outcome

• Patterns for building Test Automation Framework
• Patterns for Test Data Management, with pros and cons of each
• Patterns for managing locators / IDs for interaction with UI

Slides

Slides are available here:

Patterns in Test Automation from Anand Bagmar

Video

(My attempt of capturing the) video is available here:

Official video will be linked when it gets available.

Protractor for Angular apps?

Already asked these questions in the vodQA group on LinkedIn - but thought to repeat the same here as well - in case someone else also reads this, and has some thoughts.

I am experimenting (again) with Protractor for automation against Angular-based web-apps. This time around, my comfortness with Javascript is better (by a couple more % than before) - so I am better prepped for this challenge. 

That said, I am interested in knowing a few things on this:

• Has anyone in the group worked with protractor recently? 

• What has been your experiences in working with it? 

• Who are the roles involved in the automation implementation, execution and maintenance? 

• What are the typical utilities you built in this framework?

• How have you been modelling you page-object pattern with JS / protractor based frameworks? Or, is there some other better set of patterns for JS that should be used?

• How did you build your page objects? How did you build and manage the composition / nesting of pages? Did the method of a page return an appropriate page object?

• How many tests exist in your framework? 

• Do you run your tests in parallel?

• Do your tests run in CI? If yes, which driver do you use? Protractor site discourages the use of phantomJS. 

• Would it be possible to share some (non-confidential) examples of how you built your Page Objects? How are your specs written? Any example of that possible to see?

• Did anyone manage to run their tests against Safari / IE11 as well?

• What about soft asserts? Did you implement this?

• I saw a strange issue when running my test against chrome - I got the element is not clickable at xxx coordinates. However the same test ran against Firefox and phantomjs. Anyone seen this before?

• Given that protractor site does not recommend using phantomJS driver much, anyone used xvfb for running their tests in CI?

• What reporters do you use?

The story of a 'small' vodQA ending up being 'x-large'

We are extremely happy to start the new year with YASV (Yet Another Successful vodQA) event, this time with the theme - Agile Testing Workshop, conducted on 9th January 2016 in ThoughtWorks, Pune office.

Why the theme - "Agile Testing Workshop"?

Over the past few years, after having worked on numerous projects, interacted with a lot of clients (and their partners / vendors), and gaining insights from speaking with individuals & teams in conferences & organizations, we (the vodQA Pune team), realized that a decent portion of the Software (testing) Industry lacks decent / good understanding of Agile and effective Testing on Agile projects / teams.

So, we decided to conduct the next vodQA in Pune - focussed on Agile Testing to answer questions like - "What is Agile and what does it mean to Test on Agile projects / teams?"

Highlights

• When we started planning for this edition of vodQA, the plan was to keep it very lean - in planning, execution and participation as well. For this, we planned to keep this vodQA 'small'. Little did we realize it would end up being a patiala peg.

• What started out as an event aimed at 30 attendees soon shot up to 180+ RSVPs on Facebook to 160+ confirmations and eventually we had 85+ attendees. Including ThoughtWorkers, we (again) crossed 100+ people for vodQA Pune! - There went a lot of our 'being-lean' out of the window!

• This event was completely driven by the Facebook group (from announcements to registrations to updates).

• We had a quite a few attendees travel from out of Pune for vodQA (ex: Mumbai, Nagpur)

• This was one of the most vocal, enthusiastic and interactive audience vodQA Pune has seen. They shared their experiences and asked a lot of questions as well.

• True to our objective for this vodQA, we ensured there was sufficient time between sessions / workshops to facilitate discussions and answer specific questions from the attendees.

• We had impromptu fishbowl discussion on certain Parking Lot questions.

• After the first session of the day (Agile Game), the attendees celebrated (it was over) by bursting the balloons - early Diwali some would say … :)

• You can see the tweets, comments in the vodQA group on Facebook and pictures here.

• A huge shoutout to the organisers who were constantly tweaking their execution methods, days before the event as our expected turnout gradually rose from 30 to 100+.

Agenda and Slides

Topic	By	Slides
Welcome note	Anand Bagmar
Agile Game	Abhay Dalvi, Vardhan Bhatt & Vikrant Chauhan
Tea break
What is Agile Testing?	Amit Gundiyal & Prasad Kalgutkar	http://www.slideshare.net/vodqanite/what-is-agile-testing-56891493
Effective Strategies for Distributed Testing	Preeti Mishra	http://www.slideshare.net/vodqanite/strategies-for-distributed-testing
Lunch
Testing the Mysterious Sphere	Anjali Wadhwa, Ashwini Ingle & Preeti Mishra	http://www.slideshare.net/vodqanite/testing-the-mysterious-sphere
Break
Test Automation - Principles, Practices	Vardhan Bhatt & Vikrant Chauhan	http://www.slideshare.net/vodqanite/lessons-learnt-from-test-automation-principles-practices
Tea + Snacks break
Patterns in Test Automation (Framework + Data)	Anand Bagmar	http://www.slideshare.net/abagmar/patterns-in-test-automation

 

Feedback

• Ur event is successful when attendees r so involved that they forgot to tweet abt it till d lunch brk :) #VodQA #VodQAPune #ThoughtWorks

• Overall workshop was wonderful. Presentation and content was good. Helpful to understand and implement in our current process.

• Agile testing game taught us to focus more on quality than quantity & take feedback as soon as possible from the PO

• Though I am not working in Agile env currently, I understood whole session and got to learn something.

The always rocking vodQA Pune team!!

vodQA Pune team

Patterns in Test Automation - Framework, Data, Locators at Agile Noida

On 28th November 2015, I spoke in Agile Noida on "Patterns in Test Automation - Framework, Data, Locators".

I had spoken on the same topic (Patterns in Test Automation) in vodQA Hyderabad - but that was a Testing conference and I knew the attendees were Testing and Test Automation focused. Here, I was skeptic about how this topic would be received by the attendees - given that the conference was focused on Agile, and this topic was core Testing related, and to add to the fact - very technical where I was showing various Java / Ruby code samples.

My skepticism was, thankfully, unfounded. There was a good number of attendees that came to the talk and based on conversations after the talk, I realized that I was able to get the message across to the attendees.

Below is the abstract, slides and video from the talk.

Abstract

Building a Test Automation Framework is easy - there are so many resources / guides / blogs / etc. available to help you get started and help solve the issues you get along the journey.

However, building a "good" Test Automation Framework is not very easy. There are a lot of principles and practices you need to use, in the right context, with a good set of skills required to make the Test Automation Framework maintainable, scalable and reusable.

Design Patterns play a big role in helping achieve this goal of building a good and robust framework. 

In this talk, we will talk about, and see examples of various types of patterns you can use for:

1. Build your Test Automation Framework
2. Test Data Management
3. Locators / IDs (for finding / interacting with elements in the browser / app)

Using these patterns you will be able to build a good framework, that will help keep your tests running fast, and reliably in your CI / CD setup!

Slides

Patterns in Test Automation from Anand Bagmar

Video

Patterns in Test Automation

I spoke in vodQA Hyderabad on Sat, 22nd August 2015 about Patterns in Test Automation - Frameworks, Data & Locators.

The slides are available on SlideShare:

Patterns in Test Automation from Anand Bagmar

The video is available on YouTube:

Abstract

Building a Test Automation Framework is easy - there are so many resources / guides / blogs / etc. available to help you get started and help solve the issues you get along the journey.

However, building a "good" Test Automation Framework is not very easy. There are a lot of principles and practices you need to use, in the right context, with a good set of skills required to make the Test Automation Framework maintainable, scalable and reusable.

Design Patterns play a big role in helping achieve this goal of building a good and robust framework.

In this talk, we will talk about, and see examples of various types of patterns you can use for:

• Build your Test Automation Framework
• Test Data Management
• Locators / IDs (for finding / interacting with elements in the browser / app)

Using these patterns you will be able to build a good framework, that will help keep your tests running fast, and reliably in your CI / CD setup!

Learning outcome

• Patterns for building Test Automation Framework
• Patterns for Test Data Management, with pros and cons of each
• Patterns for managing locators / IDs for interaction with UI

Questions about the Test Pyramid

After watching my presentation on "Enabling Continuous Delivery (CD) in Enterprises with Testing", I recently got asked a couple of questions about the Test Pyramid. Thought it would be good to reply publicly - that may help others who had similar doubts. 

If you have any other questions, please reach out, or add it as comments on this post.

• Why do you talk about a JavaScript Test? I mean, why you don't consider this type of testing inside another? So, what do you mean by JavaScript test.
• JavaScript testing requires different toolset, not the standard xUnit based ones. Hence I classify it separately. Also, there is potentially a lot of logic that can be built in the JavaScript layer - so it is essential to write tests for that too - say using Jasmine.

• What's the difference between View and UI?
• UI test should focus on business / user journey validations. However a view test is different. Consider a journey which has a 5 step / screen workflow. To validate some UI change on the 4th step / screen, you will need to go through, in sequence, from step 1 to 4 and then validate the changes. This is very slow and costly approach. Instead, if you build the right type of stubs / mocks, then you can setup the state in your product which simulates the step 1-3 are completed, and directly open the UI, go to step #4, and validate your changes. This is the difference in View and UI tests.

"Agile Testing" workshop in July

[UPDATE] I think I will include some interesting Innovation Games to demonstrate the need for collaboration and more effective working. Or it could be something more specific to Testing on Agile projects. So many options ... 

---------------------------------

As part of Unicom's World Conference Next Generation Testing, I am doing a 1-day workshop on "Agile Testing", on Friday, 24th July 2015 in Bangalore. You can register for the workshop from here, or contact me for more information.

Below are the details of the workshop. 

Agile Testing

Abstract

The Agile Manifesto was published in 2001. It took the software industry a good few years to truly understand what the manifesto means, and the principles behind it. However, choosing and implementing the right set of practices to get the true value from working the Agile way has been the biggest challenge for most!

While Agile has now gone mainstream, and as we get better at the development practices to being Agile, Testing has still been lagging behind in most cases. A lot of teams are still working in the staggered fashion - with testing following next after development completed.

In this workshop, we will learn and share various principles and practices which teams should adopt to be successful in testing on Agile projects. 

Agenda

• What is Agile testing? - Learn what does it mean to Test on Agile Projects
• Effective strategies for Distributed Testing - Learn practices that help bridge the Distributed Testing gap!
• Test Automation in Agile Projects - Why? What? How? - Why is Test Automation important, and how do we implement a good, robust, scalable and maintainable Test Automation framework!
• Build the "right" regression suite using Behavior Driven Testing (BDT) - Behavior Driven Testing (BDT) is an evolved way of thinking about Testing. It helps in identifying the 'correct' scenarios, in form of user journeys, to build a good and effective (manual & automation) regression suite that validates the Business Goals.

Key Learnings for participants in this workshop

• Understand the Agile Testing Manifesto
• Learn the essential Testing practices and activities essential for teams to adopt to work in Agile way of working
• Discover techniques to do effective testing in distributed teams
• Find out how Automation plays a crucial role in Agile projects
• Learn how to build a good, robust, scalable and maintainable Functional Automation framework
• Learn, by practice, how to identify the right types of tests to automate as UI functional tests - to get quick and effective feedback

Pre-requisites

• At-least a basic working knowledge and understanding of Agile