Non Reproducible Bugs – How to deal with them ?
Last week, i had reviewed the bug reports submitted by my team & found that some of them are not reproducible. I am curious about where things went wrong & checked with the team.
Some of the Responses:
-
Look Venkat, this bug is not-producible always. So we have captured the log.
-
It does occur if we re-start the application
-
It takes good amount of time for the investigation
As a Tester, i have come across of situations where some bugs won’t reproducible again and i infer that many of the testers might have gone through the same episode atleast once.
But then, Non-Reproducible Bugs are errors of the testers even though we may not agree. More over this is true for Technical Support & Testing functions.
It’s tough for the Developers to work with such bugs and testers used to get lot of objections over such bugs. If the count is more, then it might spoil the relationships between testers and developers.
An Insight
I infer from my experiences that systems doesn’t have the powers of human brain. It’s tough to repeat the same mistake again from people since they are conscious. Systems doesn’t have that much capability interms of thinking and they do perform the sequence in which they have coded for.
How do deal with Non Producible Issues
Testers need to improve their skills inorder to reproduce such issues. The following might be a help.
-
Try to capture the story around the issue and not just steps.
-
Capture the complete Test Enviroment
-
Capture all the available Test Execution Results. This includes your test data, screen shots, applications logs, system logs, server logs
-
Need more patience (they won’t occur easily)
-
Need more observation skills on the application behavior under test.
-
Narrate the story around the issue to the stake holders and to corner the issue.
-
Resources & Time Contraints might be an issue (More true for Tech Support). Bargain for the same
Conclusion
They are bound to take lot of time & resources in the process. It’s suggested to reproduce each & every issue. We must explore more for the critical issues & re-produce them asap. In the process, we learn more too.
Happy Testing.
Explore posts in the same categories: Bug Tracking, Lessons Learned, Software Testing, Technical Support, Test Strategy, Thinking like a Tester, Training
Software Testing Blog 
May 10, 2007 at 8:31 pm
Hi Venkat!
This is MrSharma56.
What is the importance of Severity and Priority for Testing a non reproducible bug?
Also do add me up in your personal messenger preferably GMAIL.So that I feel i could get some more suggestions and comments from you.
My GMAIL ID is pavanturlapati@gmail.com
Thanks & Regards,
MrSharma56
May 11, 2007 at 10:23 am
Excellent post Venkat. Very very realistic. I have written a similar post here: http://software-testing-zone.blogspot.com/2007/05/my-top-5-ways-to-reproduce-hard-to.html . Please check this too and come up with your ideas.
Regards,
-Debasis .
May 29, 2007 at 6:52 pm
Great write up – I agree for incompleteness in the story or the supporting documents (logs, screenshots, etc) for bugs and they help in troubleshooting or investigating further. However, there are situations where the bugs appear once and becomes inconsistent in nature. Though having all the necessary information from logs, screen shots, proper sequence that producced the same bug – and at times few couple of times around the same time – however, by the time it goes to dev for a fix it is very well possible that it becomes a no-repro and trying again in a test lab may qualify the same.
Possible reasons:
1. the program sequence would have gotten fixed while fixing other issues and hence become not reproduce-able
2. once changing the lab environment on daily builds etc, any environment related constraints may become obsolete
and so on ….
The fact is, however, the issue occured and hence we have the event log information – the fact that cannot be denied as not repro and close the bug – it has to be investigated further.
What is your take on such situations and such defects?
-Ram
June 4, 2007 at 3:04 pm
Hi Ram,
Thanks for the detailed feedback.
Managing and maintaining the Test Environment is crucial for the testers.
When it comes to non – reproducble issues, we must try to reproduce the issue in the same environment. What it means is that, you need to work with the same build in which the issue is reported & make sure that the whole environment is in sync with the environment in which the issue occured.
If some tries to validate the same in latest environment the chances will be less because things might have been changed. All the information captured around the issue should help us to debug more to discover the root cause.
I do agree upon that fact that it’s tough to reproduce all the issues. But then, those are the lessons need to be learned and practiced for the testers. We might need to improve upon our skills to discover the root causes of the issue.
As i said in the post, Systems doesn’t have the power of human intelligence & we need to find the ways to repeat the same behavior.
Regards,
Venkat.
June 7, 2007 at 8:37 am
Thanks for the response, Venkat.
I know there remains tens or hundreds of issues (depending on the size of the project) remain as not-repro, would most likely miss the root cause and most likely leave bugs around that area. But again, depending on how good and supportive the program management is teams either do get to investigate further and find other bugs around or postpone the items causing some flaws in the application.
it is the trade offs some PM’s and project teams make and therby being responsible for lower quality products.
as you said, test team has the control to get the things fixed and investigated further, since they do play a role of customer advocate for quality products.
thanks
-ram
June 20, 2007 at 9:50 am
Interesting and helpful post for beginners.
June 20, 2007 at 9:51 am
Interesting and helpful post for beginners. There are certain instances that you report a bug. But that bug does not reproduce even with the same steps.
June 24, 2007 at 4:54 am
Prachi
I think, if there was a bug and that was noted with the accurate repro steps, it must repro as long as all the parameters are met (environment, hardware, software configuration, application configurations, stability, etc) — if that is not tracked or if we think that is not reproed in the same environment, then it is possible that we might have noted wrong repro steps or possibly we aren’t going through the same code path when being repro’d.
July 2, 2007 at 5:30 pm
Yes, Thanks ramsblog. I truely agree with u. I can corelate to this. There are many a times when the bug does not repro with the same steps, but then that may not be the same code path when being repro’d.
Thanks again. Comments do help beginners like me to know where we went wrong and to improve our skills.
July 9, 2007 at 11:19 am
In these kinds of cases reproducing the bugs should be easy for Developers or white box testers who knows the in and out of code. What do you say Venkat ?
July 9, 2007 at 12:32 pm
Mallik,
Knowing code will help to debug the problem better. But we may not always have access to the codebase. Based on information available for the issue, if some one can derrive the bug pattern for the issue that’s going to be great. It helps to explore other set of issues there in the bug tracker based on the bug pattern.
September 10, 2007 at 9:50 am
Venkat
Good work …keep posting this type of info …this is really very useful..
1.how to deal with non reproducible bugs> was excellent…..
Ravi
Seattle,WA
September 13, 2007 at 8:27 pm
To find bugs that hard to reproduce with embedded applications, I have found the use of relays controlled by computer programs especially helpful. When connected to various things such as power connections to really ensure that a system works reliably.
October 10, 2008 at 12:20 pm
Hi,
This is a very interesting and helpful article. I am testing mobile applications where there are times when a bug is consistently reproducible in somedays, and sometimes the same bug doesnt seem to exit. Controlling environment parameters in mobile is not possible, GPRS being the main parameter. There are time when by the same set of steps a bug is reproducible, and sometimes (TO MY DISAPPOINTMENT) it is not. I have been facing this on and off in Mobile and WAP platform related bugs. Bugs on Desktop and Web are relatively more easy to re encounter.. Please let me know about any specific steps I must follow to lower down [or nullify] the non reproducible bugs count for Mobile and WAP. Currently most of them which are not getting reproduced are kept pending with lower priority. Should non reproducilble bugs be reported or not? Even when they are high priority bugs like crash?
July 3, 2009 at 2:37 pm
Hello venkat,
Nice article, I really like it. I’m doing a bit research about test automation, and i found that macrotesting http://www.macrotesting.com be a good resource
Thanks for the article!!!!!!!!
Cheers,
Poornima
December 29, 2009 at 4:33 pm
Hi Venkat,
It’s such a nice blog. Many concepts have cleared about testing. But what if Tester have only one day and if bug is major and is not reproducible. Second day there is a release for client, so in this situation what tester have to do ?
Again Nice Article,
Hemang Kalolia
December 21, 2010 at 10:28 pm
nice job guys,i like your simple themes.
Thanks a lot for nice article.
January 5, 2017 at 10:32 am
Hi, thank you for this post I agree with you that Non-Reproducible Bugs are errors of the testers even though we may not agree. Moreover this is true for Technical Support & Testing functions. very useful information
February 22, 2017 at 10:10 am
Hi, thank you for this post I agree with you that As a Tester, I have come across of situations where some bugs won’t reproducible again and I infer that many of the testers might have gone through the same episode at least once. very useful information
March 31, 2018 at 2:09 pm
[…] Non Reproducible Bugs – How to deal with them ? […]