Archive for the ‘Lessons Learned’ category

Non linear paths from Application Code

October 4, 2007
The applications become complex as their code base increases. This has challenges for the testers to determine the nonlinear paths in the application.

Most of the Static Analysis tools over the application code helps us to  identify the  cyclomatic complexity (nonlinear paths) at a method level. These might be helpful to validate those methods and to achieve good code coverage over the same.

But the Code coverage at a Unit Level may not be a big help since most of the end user scenarios won’t run after unit level paths. These paths is an integration of the above unit level paths.

Since the Testers focus on simulating the end user scenarios, it will be good to identify all the possible nonlinear paths around the application code base and capture the code coverage based on these paths.

You might want to go through some discussion around this on Linkedin Answers

In case you have similar experiences over white box testing drop me a mail at venkatreddyc@gmail.com

Happy Testing…

Powered by ScribeFire.

Advertisements

Ability to identify the hot spots of release from Bug Database

September 24, 2007

Bug Database for the products might have thousands of issues over a period of time against various builds and releases. Though these issues fixed over a period of time, it might be hard to derive meaningful metrics over the release.

We need to support these releases over the production systems & it might be helpful to capture the hotspots / risk elements with the release. Most of the issues here to deal with the respective features, compatibility with other features / technologies & performance related issues.

The usual metrics of number of issues against a module and their severity levels may not be of help always.

How easy is it to derive the following from the Bug Database for a given release

  1. Identify the issues have originated from Requirements, Design & Implementation
  2. Identify the issues over their category (Functional, Performance, Security, Compatibility, Usability etc)
  3. Identify the issues along with their origin & category over the features rather than the modules / components

Powered by ScribeFire.

The Life Cycle of a Bug – Different Stages in it.

September 10, 2007

In this post, i will explore  different stages of the Bug from it’s inception to closer. The Bug has been found and logged into the Bug Tracking System. It’s my fourth post in the Bug Life Cycle series.

  1. The Bug has been found and logged into the Bug Tracking System. It will be treated as New Bug in the System.
  2. The Bug will be assigned to the concerned Developer for a Resolution.
  3. The developer looks in to the possibilities of the resoultion & takes a call on Resolution by fixing it or differing over the information provided.
  4. Tester validates the resolved issue in the build & checks for the regression scenarios over the fix.
  5. If the issue found fixed, then he choose to Close the issue else he / she will Re-open the same.
  6. The Cycle follows again for the re-opened issue till it get’s closed.

Bug Life Cycle

It worth doing the following activities

  1. Capturing the required and re-usable info to the Bug Report at it’s each stage.
  2. Check for all the closed bugs of Severity 1 & 2 against final build for the release.

In the next post, I will share my thoughts on the useful metrics over the Bug Tracking Repository.

Happy Testing..

Whitebox Testing – Is it really white ?

September 7, 2007

The popular myths around Blackbox & Whitebox Testing are by it’s name. It’s black since we can’t see it (don’t have access to the code) & it’s white since you have access to all the code. But then, With in the code there are many black boxes inside and it’s tough to have access to that code base.

  • We don’t have access to code of a language API. Most of the applications have been built on top of a API & assume that the API works fine
  • Most of the application do integrate some third party tools over it’s API. We don’t have access to that code base.
  • We don’t have access to the code of Compiler
  • We don’t have access to code of rum time engine that executes our application code
  • We don’t have access to the code of Operating System Services on top of which the application runs

The list goes on and there are many black boxes in side our code too. We are just testing the code written for the application and it’s better to call it as Code Based Testing rather than Whitebox Testing

— Happy Testing..

Bug in the BSNL Portal

August 10, 2007

Today, i have come across an interesting issue with BSNL Portal. I have been using this site for online billing and payments for about an year.

Followed the usual steps and my objective is too see the bill and pay it online. To my surprise, the system says

This error message ” Invalid object name ‘RECEIPT_MASTER’ “ is strange and there is no information for me on how do i proceed further. The application might be unable to get the specified object from the Database.

In case if this happens on the last day of the payment, on line users are liable for late payment charges due to this issue.

Users are familiar with the messages like, Unable to process the information as of now & please check back after some time.

The error messages must be meaningful to the business users and we need to make sure that the technology related terms / phrases are not part of the same.

Happy Testing…

Powered by ScribeFire.

Context driven information for Bug Reports

July 26, 2007

Context driven information is the need of the hour and there is a huge value associated with the same. It’s good to capture the context driven information in the bug reports. My initial experiences with bug reports way back in early 2000 have taught many lessons to improve upon.

Bug reports used to capture what is the problem with the system familiar to the user (tester) who reported the same. People spend very less time to capture all the details required and there are many reasons for the same. I hope some of the upcoming testers will also be in the similar situations.

Some of the Reasons people quote here

  • We need to test more and less time to capture & write more information in the Bug Reports
  • It’s tough to capture all the information required
  • System is complex & It’s tough for the novice users to understand the bug reports
  • You know capturing all the info is process driven & it may not be worth of efforts
  • It some times boring activity to collate the info & push it
  • I can reproduce it on my machine if developer needs it.

This list can go on…

I hope you have come across this situation at least once in your career.

This is my third post in the Bug Life Cycle Series and it’s good to know the different users of your Bugs and their context with them. The mission of your bug report is to provide details and context of the problem and convey the importance of it with a user driven stories.

Your bug report must be the voice of customer and it need play the role of an advocate against the problem. Bug Advocacy from Cem Kaner is an excellent source to begin with. If the bug report unable to specify the need of the context, then it’s better not to write any report

It’s good to explore & capture some of the following problems

  • Productivity
  • Performance
  • Usability
  • Migration
  • Stability etc

Try to link your issues with most suited functions listed above. It may not be obvious to other users in the system to explore & analyze the issues in that fashion.

There is another context associated with Bug Reports. That’s with the stake holders of the project. The Bug Tracking system must give the right trends and identify the hot spots. Testers must capture the right kind of data to derive better valuable metrics over the bug repository.

Care must be taken to capture

  • Capture all the Test Environment details
  • Detailed classification on the feature. Classify to the maximum possible sub feature/component of the system
  • Clarity on Severity & Priority
  • Versions and Build Numbers (Affected & Fixed)
  • Bug Classification (Requirements / Design / Implementation etc)
  • Bug Types (Functional, Performance, Usability, Security etc)
  • It can go on…

The above info helps a lot to identify the trends in bugs and focus on the unstable components / environments.

Final Thoughts
Push the entire context driven information to the bug repository at least for a release cycle and observe the results. Check back with your repository to identify the trends and risk associated with the release and I am sure that it will be in the similar lines of end user feedback.

Happy Testing…

Use Cyclomatic Complexity to determine the Risk and Test Scenarios

July 19, 2007

Cyclomatic Complexity (CC) is a software metric (mostly code based metric) used to the number of independent path executions in the application. Introduced by Thomas McCabe in 1976, it measures the number of linearly-independent paths through a program module.

It helps the developers to determine the independent path executions and base line unit tests that they need to validate. Using this, the developers can assure that all the paths have been tested atleast once. It’s a great comfort for the developer and their respective managers.

It’s better to write JUnit Tests for all these linearly-independent paths and integrate it with any code coverage tool. These reports help to focus more on the un covered paths and improve the code coverage.

It also helps to evaluate the risk associated with the application. The following are the results published by SEI and they are being followed widely to determine the health of the code base.

Cyclomatic Complexity      Risk Evaluation
1-10                                              A simple program, without much risk
11-20                                            More complex, moderate risk
21-50                                            Complex, high risk program
Greater than 50                          Un testable program (very high risk)

Explore more at Cyclomatic Complexity in Software Technology Roadmap from SEI.

Further Reading on the topic

Use metrics to evaluate the risk early in the cycle & improve your test coverage.

Happy Testting …


%d bloggers like this: