vodQA - The Saga Continues in Pune!

After a long break, vodQA returns to Pune. This time, ThoughtWorks & Vuclip are jointly hosting vodQA.

At vodQA, we have always strived to focus on the art and practice of Testing. This edition of vodQA is no different. Hence the theme for this vodQA - "The Saga Continues". 

We welcome all roles interested in the (Software) Quality to participate in this conference.

The event will be held on Sat, 29th July 2017 at Vuclip office in Pune. 

Agenda is as below:

Address:

1st Floor, 

Nanasaheb Gaikwad Information Technology Park, 

Sarjaa Rd, Aundh, Pune, Maharashtra 411007

Above Croma

Link to Map - https://goo.gl/maps/4yWaX6atutw

Changing logcat buffer size in Android devices ... almost works

My (debug-build of) app under test logs extra information about test execution to system logs which is accessible via logcat on Android devices. This is very powerful as now I can run my cucumber-jvm / Appium tests, copy the logcat file after the test execution completes, parse it for relevant information, and do appropriate assertions on the same.

The default buffer size on Android devices I have seen is 256kb. This is less for me - as I end up losing the earlier information, and hence my assertions fail.

Thankfully, there is a programmatic way to change the logcat buffer size in the device before running tests. The command is

adb logcat -G 3M

This adb command works in the Motorola devices in my MAD LAB, but does not work in Samsung devices. The error I see on running the above command is "failed to set the log size"

Any idea why this would not work in Samsung devices? or rather, what do I need to do to change the logcat buffer size?

[UPDATE] - Interestingly - this works on Samsung Galaxy S7, but NOT in Samsung J5 Prime OR Samsung J7 Prime

Criteria for setting up a Mobile Test Automation LAB

I recently got asked this question related to the MAD LAB (Mobile Automation Devices LAB) - "Would like to understand how can we setup something similar in our organisation?"

Since this question is applicable for all those thinking of, or have already set up their own lab, thought I would share my answer here.

To setup your own LAB for Mobile Test Automation, multiple things need to align:

Supportive management who -

• allows experiments (within reason of course) and encourages learning through failure, 
• willing to invest in infrastructure ($$)

Skilled and Passionate team members who -

• understand the domain well, 
• willing to learn, experiment, re-learn and fail fast, 
• keep looking for innovative solutions to solve problems on hand, 
• do not reinvent the wheel. 

Philosophy aside, our MAD LAB has the following: 

• Mac Minis (8-12 devices per Mac Mini), 
• Powered USB Hubs (I use the ones shown below - and they are working pretty well)

• High-quality USB cables (I use the ones shown below - and they are working pretty well)

• CI (Jenkins) setup correctly to keep running tests continuously, proper reporting  in place (else whats the use of running tests if you do not look at the results)

You could start with similar IF it fits your product-under-test context

After I answered this on LinkedIn, I realised, there are more parameters to think about, than just the above.

• Knowing which devices to use in your Lab
• Having good, reliable Internet connection
• Devices should be "seen" easily
• Should be easy to work on / with the devices as and when required
• Know how you the devices will be placed in the lab. We tried the following:
• 2-way tape - that didn't work. Devices used to stay up for a few days, then "drop" suddenly. Of course, that also depends on the back surface of the devices.
• We tried many mobile stands / hangers (shown below) - but each had their own limitations

• Finally I found an industrial-strength velcro (1" velcro tape that could take a couple of pounds of weight) - and my devices have not budged since. PS: Please be careful when putting on this velcro on the devices. IF it gets on your hand, you will have a velcro tattoo for a long long time.

What other parameters would you consider for setting up your own Lab? Looking forward to the comments below.

Introducing MAD LAB - for Mobile Automation

The past few months I have been heads-down in stabilising my Real-Device Mobile Test Lab - which we now call MAD LAB (Mobile Automation Devices LAB) .

For those who may not recollect, see my past posts for reference -

• A new beginning - entertainment on mobile

• How to enable seamless running of appium tests on developer machines?

• Features of my Android Test Automation Framework

• How to upgrade the appium-uiautomator2-driver version for appium 1.6.3?

• Finding my way out of bottomless pit with Appium & Android 7.0 for parallel test runs

• Sharing implementation of cucumber-jvm - Appium test framework

Along with my colleagues, we have put in lot of effort in setting up MAD LAB and have now added a lot of rich features to help running tests, seeing the results and making sense out of them easier. 

• All infrastructure management is implemented now in groovy (instead of gradle as shared earlier).

• All automatic test execution is managed via Jenkins file

• Actual test implementation is done in cucumber-jvm / java

List of features currently implemented:

• Device management (selection, cleanup, app install and uninstall)

• Parallel test execution (at Cucumber scenario level) - maximising device utilisation)

• Appium server management

• Adb utilities 

• Managing periodic ADB server disconnects

• Custom reporting using cucumber-reports

• Video recording of each scenario and embedding in the custom reports

Contents of MAD LAB:

• 1 Mac Minis - running various Jenkins Agents

• 2 Powered USB hubs

• 8 Android devices

Here are some pictures from the setup.

There are many more features, in various stages of implementation, being added to make MAD LAB more powerful.

Sneak peek into whats coming:

• Analytics Testing

• Trend and Failure Analysis 

• iOS

• Web

• A transformed MAD LAB

Finding MAD LAB interesting? Some very interesting changes are coming in soon. Watch out for my next blog post for that. 

Want to contribute and be part of this journey? Even better! Reach out to me!