Test Execution Trend feature added in TTA (0.5.7)

A new version of Test Trend Analyzer (TTA) is now available on github.

These are the new features in it:

• Test Execution Trend - to see the benchmarking of specific test execution over a time period

• Failure Analysis View

• Modified view to show failures for specific Test category along with specific test run

• Modified view to show failures for All test categories with latest test run

• Minor Refactoring for Compare Runs view

• Query optimisations + minor defect fixes.

For those that missed the earlier announcement - summary of features of TTA is available here, and source code is available on github.

PS: There is a lot of work (features / UI) planned for TTA. If you wish to contribute, we would love your help!!

TTA (0.5.4) feature information and screenshots

Here are some screenshots from the current functionality supported by TTA v0.5.4.

See the TTA Wiki page or Contact me for more information on TTA.

Manual upload of Test data

Test Pyramid

Comparative Analysis

Compare different test runs

Test Failure Analysis

Integrate External Dashboards with TTA

TTA Statistics page

TTA - closer to becoming unified dashboard for Test Trends and Status

Test Trend Analyzer (TTA), went live again today !!! We are now at version 0.5.4.

In the past week we have made a bunch of improvements and added new features - Integrating External Dashboard , which takes TTA closer towards being the central dashboard for testing status across the organization.

 

There are a couple of projects within ThoughtWorks using this, and also some other organizations.

Here is the list of new Features:

• Integration of external dashboards (add from /admin page, see integration on /home page) - this allows one to integrate different existing dashboards into TTA - to make it a one stop place for seeing all Testing related information. Example: You can integrate your defect reports from Mingle / Jira / etc., or, you can also integrate your specific Continuous Integration (CI) dashboard from Go / Jenkins / Hudson / Bamboo / etc.
  
• Compare test runs (/compare_runs) - to compare specific test runs --- 
• what are the common failures, 
• what are the unique failures, 
• what failed on date 1, but passed on date 2
• what failed on date 2, but passed on date 1
• TTA Statistics Page (/stats) - to know usage of TTA by different projects / teams in your organization
• Fixes + minor UI modification

Features available from some time:

• Test Pyramid view (/pyramid) - to see how your project's automation effort aligns with the Test Automation philosophy
• Comparative Analysis view (comparative_analysis) - to see the trend of your test automation results over a period of time, and if any patterns emerge
• Failure Analysis view (/defect_analysis) - to make better meaning of the test failures, and help you prioritize which failures should be fixed first.
• Upload Test Run Data manually (/upload) - to manually upload test data in case if you have not uploaded test data automatically to TTA, but still want to use TTA

 

More information about features, how to use TTA, etc. can be found on this blog, TTA-github, TTA-github-wiki, or by contacting me.

Introducing Test Trend Analyzer (TTA)

The statement "I have a dream" is a very famous quote by American activist Martin Luther King Jr.

I resonate very closely with that. Here is why and how ...

Almost 2 years ago, I had a dream ... a vision about a product that can help those working in large organizations understand the health of their products / projects at a quick glance, instead of having a team of people manually scrambling frantically to collate and collect the information needed to get a sense of quality about the products they support. I chose to call this product Test Trend Analyzer - TTA

Given that Automation is a key factor in ensuring the success, quality and time-to-market for products, I took that as a baseline requirement and came up with a basic high-level diagram for TTA:

TTA - first diagram

Finally, a couple of months ago, I found a bunch of passionate people, who also had some time, to try and implement this tool.

We came up with this vision for TTA:

TTA Vision

Accordingly, we planned, prioritized, spiked, did some prototypes, did demos and showcases and got a base usable product completed.
This is a open-source project, available on github with more information about it on the github wiki.

The technology stack used is: RoR with mysql db. 

TTA falls in the Big Data + Visualization space - specific to Testing.




Watch this space for more information about TTA and the currently supported features. Email out to me if you need more specific information, or have questions on how can you use TTA, etc.

How can you contribute?

Given that we have implemented only a few basic features right now, and there are many more in the backlog, here is how you can help:

• Suggest new ideas / features that will help make TTA better
• Use TTA on your project and provide feedback
• More importantly, help in implementing these features

Test Trend Analyzer (TTA)

There are many tools and utilities that provide ways to do test result reporting and analysis of those results. However, I have not found a good, generic way of doing some Trend Analysis of those results. 

Why do I need to Trend Analysis of the test results?

Long(er) duration projects / enterprise products need to know the state of the quality of the product over time. One also may need to know various other metrics around the testing - like number of tests, pass %, failure %, etc. over time.

The reports I have seen are very good about analyzing the current test results. However, I have not really come across a good generic tool that can be used in most environments for the Test Trend Analysis over a period of time.

I am thinking about developing a tool which can address this space. I call this - the Test Trend Analyzer (TTA).

Here is what I think TTA should do:

Supports:

• Work with reports generated by common unit-test frameworks (jUnit, nUnit, TestNG, TestUnit, style of reports)
• Provides Web Service interface to upload results to TTA
• Test Results uploaded will be stored in db
• Will work on Windows and Linux

Dashboard:

• Creates default TTA dashboard
• Customizable TTA dashboard
• Dashboard will be accessible via the browser

My questions to you:

• Do you think this will help? 
• What other features would you like to see in TTA?
• What other type of support would you like to have in TTA?