Archive for March 2007

Taking on “Testing Triangles – A classic excercise”

March 24, 2007

Solving puzzles, sudokus & the analytical problems is one of my hobby. This set of puzzles always helps me with some brain food which mandatory to stay tuned in the current trends.

I was going through this Testing Triangles post from Elisabeth Hendrickson from Quality Tree Software. The triangle program is very much popular to the world of testers and many of the books and blogs have already covered the same.

Elisabeth posted this as a testing puzzle and asks the people to uncover the bugs in her program. The beauty of this Triangle program is that whole code is available for the tester since it’s written using JavaScript.

I couldn’t stop my self exploring the program and discovering some issues with Testing Triangles and logging them into comments sections while in hurry too.

I thought for a while on this program in the evening and felt that there are many similarities for this even though it’s very very small when compared to the real world applications we test today. So let me take a re-look at this program and apply my test approach for the problem.

Step1: What is the Problem ?

We need to test a program that takes the three sides of a triangle and tell the user about what kind of triangle is it. Here is that Triangle program which has been written in Java Script.

Find the description from the post here

This version of the program takes as input three numbers representing the size of the sides of a triangle. When the user clicks “Draw” the program draws a picture of the triangle with the size of the sides shown in proportion and also displays the type of triangle.

Step2: Question the problem & get the required information for the Test.

The Testers first need to understand the application and the scenarios in which the same might be used.

So with this perception, let’s re-look at the information available for the application. This is true that the above information provided for the application, acts as Requirements, User Manual and any other info we can call it as.

Like many applications that we test now, even this doesn’t have clarity on the information provided. This is also one of the reason for me to call it as a classic example for testing. If we dig more into it the clarity on the information is missing and there are lot of implied requirements (at least from the developer perspective) /assumptions made by the developer.

It’s not clear from the above description that

  • What is a Triangle and how to define the same ?
  • How many types of triangles are there and how to define each one of them ?
  • Are there any boundary limits for the size of the sides that makes triangle ?
  • Can any given three numbers make a triangle ?
  • etc..

Since the info is not available and the Triangles are very generic. Let’s try to Google around the same and get some info to explore the application.

So get the info here, here, here, and here

Step 3: Sanity / Smoke Test for Triangle Program

By now, I know what a triangle is to begin with and how the same can be drawn. Let me do some sanity checks here.

  • For the Test Data of 1, 1, 1 as the values for three edges of triangle the output of the application is “Triangle Type: Equilateral”
  • For the Test Data of 2, 2, 3 as the values for three edges of triangle the output of the application is “Triangle Type: Isosceles”
  • For the Test Data of 5, 6, 7 as the values for three edges of triangle the output of the application is “Triangle Type: Scalene”

 

I have started with some very basic positive tests to make sure that the application is inline as per requirements. It’s not good to test the application against negative tests to begin with.

 

Let’s explore some more tests.

  • Try with 0,1,2 – Invalid – wow // might have checked for 0
  • Try with -2,3,4 – Invalid – wow // might have checked for -ve values
  • Try with 1,2,3 – Invalid – // what ? why is this invalid

For the Test Data of 1, 2, 3 as the values for three edges of triangle the output of the application is “ Triangle Type: Invalid

The Application says it’s invalid. But as per the above spec it should be a Scalene Triangle since all the edges are different from each other.

The following queries came to my mind

  • Is this a bug with Application ?
  • Am I missing any more info that’s required for the test ?
  • Can any given three edges make a triangle ?
  • Is there any logic behind the same ?
  • But it could have been good,if the application says why the data is invalid. I have no clue on this and it’s a usability issue.

 

The general tendency that i see in many of the testers is to report it as a bug the moment they see some different behavior with the application and run into many discussion over the same. I have already blogged on it with a post as That’s Expected Behavior and not a Bug. A little more depth into information gathering will be more effective.

Let’s see why this test data is invalid for the application.

I got some more information again over the web on the edges of the triangle that will help me to understand the issue. And this is the logic behind the same. Get more info on this here.

In order to construct a triangle, the sum of any two edges must be greater than the third one.

 

So with the above data of 1,2,3 we can’t construct a triangle and the data is invalid. But a error message here on why this is invalid from the application could have saved us lot of time in exploring the same.

One more test with 3,4,5

For the Test Data of 1, 2, 3 as the values for three edges of triangle the output of the application is “ Triangle Type: Right”

 

Oh the behavior is different again. i am expecting of Scalene Triangle. Some more info via message could have been helpful for users on why the same has been labeled as that particular type. Might be the developer expecting all the users to have the domain knowledge.

 

Step4: Thumb Rules of the game

By now, we have capture a bit more info on the application & let’s list the same here

  • Rule 1 : Sum of all the angles inside a triangle must be 180 degrees.
  • Rule 2 : In order to construct a triangle, the sum of any two edges must be greater than the third one.
  • In case of sides, they are called as equilateral triangle, isosceles triangle, right angled and scalene triangle
  • In case of angles, they are called as right-angled triangle, obtuse triangle and cute triangle

 

It’s a good practice to update this section as and when we get the more info about the app behavior & fine tune the same.

 

Step5 : Black Box Testing on the Application

We have the following info as of now through above test observations.

  • Application doesn’t have any validations for the input value to make sure weather it’s a number or not .
  • The is at least one test case that has passed for all the types of triangles

Many testers with so much information available now for testing still trying to attack on the system with negative tests (people used to call it as negative testing) to uncover some issues on the first phase itself. Even though we uncover many issues there it won’t be worth trying out the same without exploring the basic functionality of the application.

Let’s explore the application with some more Tests

The side of the triangle must be a numeric value. So the application should allow only that kind of data. Application must draw the triangle with in the area given for the same.

  • Try with 6,8,11 – draws out side the box
  • Try with 3,4,5 – Right Angled
  • Try with 6,8,10 – Scalene- // Why so. why can’t it be right angled ?
  • Try with 3,5,10 – Scalene // what ? Basic violation of thumb rules. it must be invalid. more over it just draws a single line and not a triangle
  • Try with 6,7,15 – same as above

There are couple of critical bugs here. For an input data which supposed to be invalid, treated as a triangle and other one is that it draws the triangle out of the box.

So there must be some goof up happened while handling the conditions on input data to satisfy them as valid or invalid. This has been proved now with the way that the above tests revealed the application behavior.

Let’s do some code based testing now since it’s a java script and the whole code is available.

Step6: White Box testing on the Application

Let’s look at some important piece of the code.

if (this.s1 >= this.s2 + this.s3) {

 return "Invalid";

}if ((this.s3 == this.s2) && (this.s2 == this.s1)) {

 return "Equilateral";

}

if (Math.pow(this.s1, 2) == (Math.pow(this.s2, 2) + (Math.pow(this.s3, 2)))) {

 return "Right";

}

if ((this.s3 != this.s2) && (this.s2 != this.s1)) {

 return "Scalene";

} else {

 return "Isosceles";

}

The above code helps us to get the type of triangle based on the input data. Some observations from the above code

  • If there are no validations for the input values, then it’s going to be a major problem. Think of the scenarios like (0,0,0) or (-2,3,4)
  • For this (this.s1 >= this.s2 + this.s3) condition we need to assume that the input data must be sorted from max to min being the s1 as max value and s3 as min value. Else this is going to be big blowup

Look at this bunch:

Triangle.prototype.setSides = function(s1, s2, s3) {
	sides = [s1, s2, s3];

 sides = sides.sort();	this.s1 = Number(sides[2]);

 this.s2 = Number(sides[1]);

 this.s3 = Number(sides[0]);

this.scale = 310/this.s1;

 this.scaledSide1 = this.scale * this.s1;

 this.scaledSide2 = this.scale * this.s2;

 this.scaledSide3 = this.scale * this.s3;

};

It’s reading the input data, sort it and assign the same for the values for the edges. I am concerned about the sort() method here. It’s a block box for us and need to explore on what it does. Other piece of does simple operations.

Let’s ask the google again for some help here. Find some important reading here on the sort method.

Oh my god. This is the problem. It’s doing a generic sort and not tuned towards the numeric sort. So this is the root cause of all the issues. It’s such an issue that even a programmer might over look the same by it’s nature

Find some info on how JavaScript sort works below from the link

Example

In this example we will create an array containing numbers and sort it:

<script type="text/javascript">
var arr = new Array(6)arr[0] = "10"arr[1] = "5"

arr[2] = "40"

arr[3] = "25"

arr[4] = "1000"

arr[5] = "1"
document.write(arr + "<br />")document.write(arr.sort())
</script>

The output of the code above will be:

10,5,40,25,1000,11,10,1000,25,40,5

Note that the numbers above are NOT sorted correctly (by numeric value). To solve this problem, we must add a function that handles this problem:

<script type="text/javascript">
function sortNumber(a,b){return a - b

}
var arr = new Array(6)arr[0] = "10"arr[1] = "5"

arr[2] = "40"

arr[3] = "25"

arr[4] = "1000"

arr[5] = "1"
document.write(arr + "<br />")document.write(arr.sort(sortNumber))
</script>

The output of the code above will be:

10,5,40,25,1000,11,5,10,25,40,1000


Step7: Summary:

I have taken a this approach for this application since it’s simple to apply & explore in this context. It’s good to always start exploring the applications by gathering information from all walks.

Keep getting the info, fine tune your tests and observe the application behavior. Set the base ground right and dig more and keep digging. You are bound to uncover all the important issues with the application.

Advertisements

Role of a Tester – An interesting discussion with an upcoming tester.

March 23, 2007

Today i had an interesting discussion on the role of the testing with an upcoming tester Debasis from Chennai, in India.

Debasis: Hi venkat

Venkat: what’s up debasis

Venkat: going great. Happy Testing and more posts on my blog these days.

Venkat: That’s interesting. let me ask you some stuff on Testing

Venkat: what’s the role of testing in SDLC

Debasis:The same as of parents in the upbringing of a child

Venkat: oh. That’s interesting. But don’t you see it’s the role of the management more

Debasis: Okay b4 I cud answer, i want ur definition of SDLC

Venkat: The process used to develop software the phases involved init

Debasis: But to me the processes can be of many types and of different approaches. Assuming that I am following a generalised SDLC approach…

Venkat: go ahead.

Debasis: The role of testing is to ensure that the intended user gets a good quality product after the release

Venkat: hum. good quality product. great go on.

Debasis: But that in no way mean that the Product is Completely bug free after testing

Venkat: So what do you mean by a good quality product then. Are you acting as a quality police in order to get the good quality product out.

Debasis: what do you mean by quality police

Venkat:Look at this post to get some info on quality police

Debasis: oh no. I have not seen a quality police in my entire career. There may be ppl claiming to be quality police

Debasis: But my role is to point out important defects as quickly possible in a given SDLC phase

Venkat: that’s great to hear and expecting the same. u r coming on the track now. This is the role of testing.

Debasis: Thanks…

Venkat: Catching up important defects in early stages helps a lot. This is how you can become the headlights of the project as James Bach used to say by giving the right information at the right time.

Debasis: Yes u r right

Debasis: And to try and get them fixed b4 a Product release

Venkat: hum wait a while let’s discuss on fixing the issues part at a later stage.

Debasis: oh ok.

Venkat: How do you know that the bug that has been discovered is critical or not.

Debasis: Simple. If the defect is such that which may/can bother the stake holders of the product, them I assume that its important.

Venkat: Are you assuming ? Will there be any evidence for this bias.

Debasis: might be checking with the docs and stakeholder will resolve the conflict.

Venkat: That’s good to know.

 

Venkat: let me come back to this. Are you trying and get the issues fixed b4 a Product release

Debasis: yes. It’s my job to get all the issues fixed that are reported by me.

Venkat: Does your bug reports clear for the deveper to re-produce the same.

Debasis:Yes there are really good.

Venkat: Does your bug reports contain complete information on the bug. A story about the same and helps developers to reduce their debugging time for the issue.

Debasis: And I as far as reproducibility of an issue is concerned, I am thankfull to god to have been gifted with gr8 bug reporting skills 🙂

Debasis: Also sometimes I am assigned to try n reproduce some hard to reproduce issues posted by other testers

Venkat: that’s great man. So what do you mean by trying to get the issues fixed.

Debasis: Convincing, motivating & helping the developers to get the issues fixed.

Venkat: oh is it. Why do you need to do it. Let me ask you some thing different. How many times did the developers tried you to convince, motivate and helped you to uncover more issues in their code. Think for a while on this.

Debasis: What ?

Debasis: Hmm.. May be u r rite. Will definitely think over it

Venkat: you have given the enough information on the issues so that they can be re-produced or to debug the same. And more over you have also given it’s relevance wrt it’s stake holders.

Debasis: But isn’t my job to get my issues FIXED? If they r really imp.

Venkat: No way. That’s the role of the management. If they are really imp, they will get it fixed. Why should u worry about the same and spoil the relationships with the developers.

Debasis: hmm, good point. will try to ponder about it

Venkat: most of the testers here in india see Testing as a support function to the development. So this has been there in their mindset.

Debasis: But I don’t. I see Testing as an important part of SDLC which can’t be substituted by anythingelse

Venkat: That’s great to hear. If that’s the case, i don’t expect the statement, But isn’t my job to get my issues FIXED?. how many times did ur dev team asked to report bugs on their code ?

Debasis: May be u r rite. Will definitely think over it

Venkat: The role of tester is to catch the imp bugs at the early stages and give complete info on the same as qucikly as possible. It should contain all the required info for that issue to reproduce.

Debasis: Sure, thanks. Any way, I enjoy chatting with you

Venkat: i want an upcoming testers like u to be very clear on the role of testing and the role of tester. Once this is clear, you can explore on the skills that are required for that role and improve the same. hope this helps

Debasis: Thanks again for sharing ur views on testing with me

 

Enjoy the reading and share your views

 

 

JavaNCSS – A source metric suite for Java

March 15, 2007

I have been looking after LOC counters to capture some metrics of the source code and found that JavaNCSS is a good match.

What is JavaNCSS ?

JavaNCSS is a command line utility which measures some standard source code metrics for the Java programming language. The metrics are collected globally, for each class and/or for each function.

What are the advantages ?

The following are some of the advantages that i have seen

  1. Support for Ant Tasks, so easy to integrate with build process
  2. Reports can be in Text, XML, HTML etc
  3. Support for Stylsheets and easy to get nice HTML reports
  4. Metrics at each level Package / Class / Method
  5. Cyclomatic Complexity Number
  6. List the number of packages / classes / functions / LOC counter at each level

Further Reading:

  1. JavaNCSS Home
  2. LOC Counters for C++ / Java on Joel Software Discussion group
  3. SLOC on Wikipedia

Testing Magazines

March 14, 2007

Testing Magazines helps us to know the latest trends / happenings around the industry. These will list the methodologies, tools, techniques, expert talks and some of the best practices that are useful to the testing community.

Through this post, I would like to list the magazines that are useful. I have been following through the below list of magazines since a while and these does helped a lot.

Software Test and Performance from BZ Media

Brought to you by BZ Media LLC, Software Test & Performance helps readers improve the efficiency of their software QA and testing processes, as well as help them improve the performance of their in-house applications pre- and post-deployment.

Better Software from StickyMinds.

Better Software is the magazine for software professionals who care about quality. Each issue brings you relevant, timely information to help you build better software. Continuing to deliver in-depth articles on testing, tools, defect tracking, metrics, and management, it’s the only commercial magazine exclusively dedicated to software professionals. Within the pages of each issue you’ll find heavy hitting articles about solutions to common management problems, coverage on emerging technologies, and more. You’ll benefit from expert analysis and real-world case studies in the areas of Testing & Analysis, Managing People & Teams, and Tools & Techniques.

Methods & Tools

This is a free magazine with PDF and text issues that provide practical knowledge and information on all topics of software development and software engineering: UML, Agile Methodologies (eXtreme Programming, Scrum, TDD, FDD,..), Software Testing, Software Configuration Management, Database Modeling, Java, .NET, RUP, Software Project Planning and Management, Test Automation, Programming, Software Analysis and Design, Quality Assurance, Software Process Assessment and Improvement, Software Development Tools, Risk Management, Refactoring, IT News, etc.

The Professional Tester

Professional Tester is the leading publication targeting professional testers of software worldwide. It is a 32-page, English-language quarterly magazine with a circulation of approximately 15,000 subscribers and is now in its sixth year of publication. Subscription is free to qualifying industry professionals. Professional Tester is completely vendor-independent and is currently the only European publication covering the rapidly growing software testing market.

The Tester from Specialist Group in Software Testing

WINNER of the BCS Best Specialist Groups Magazine award!. The Tester is the newsletter of the SIGIST. It is published a few weeks before each conference and contains advanced details of the conference as well as a booking form.

Tester is not a Quality Police

March 9, 2007

The perception from most of the management is that the Testers are the quality police for the products / projects that they develop. This is true for most of the companies where there is no Quality Assurance group.

What do i mean by a Quality Police ?

  1. Is the requirements freeze done for the product
  2. Did we write all the specs like SRS, Design Docs, Uses Cases
  3. Are the developers writing Unit Tests ?
  4. Are the developers testing the builds before pass on the same to Test
  5. This list goes on…

There are testers who are trying to do the above and i call them as Quality Police .

Where does this attitude come from ?

The attitude of acting like a Quality Police for testers is injected by the management from the beginning of their careers. So they expects the same as they go along with their career and this attitude spoils the relationships between developers and testers.

Acting like Quality Police and trying to get things done is more of process which need to be followed by all the teams and let’s not take that up. Let the management take care of the process management too.

What is Quality ?

Wikipedia describes quality assurance as follows

Quality Assurance is the activity of providing evidence needed to establish confidence among all concerned, that the quality-related activities are being performed effectively. All those planned or systematic actions necessary to provide adequate confidence that a product or service will satisfy given requirements for quality. Quality Assurance is a part and consistent pair of quality management proving fact-based external confidence to customers and other stakeholders that product meets needs, expectations, and other requirements. QA (quality assurance) assures the existence and effectiveness of procedures that attempt to make sure – in advance – that the expected levels of quality will be reached.

Read more on Quality Assurance from wikipedia

It’s worth noting here that Good Quality doesn’t come from less bugs in the product. I have seen people whose assumption is that if a product contains bugs it’s of less quality and in case if it doesn’t it’s of more quality. I would say that we can’t measure the quality of the products just with Bugs. I have never seen any product which doesn’t have bugs in it.

Are the Testers responsible for the Product Quality ?

The Testers test the product and report bugs. Some of these bugs will get fixed for the release in case if they are import. You can’t build quality into systems just by reporting bugs and helping developers to fix them.

Assuring the quality into Products under development is the responsibility of all the teams associated with Product Development

So What should be the Role of Tester then

The role of the Tester to test software, find bugs, report them so that they can be fixed. The Bug Reports should be clear, easy to reproduce, reduce the time to debug the same for developers and the report should motivate the developers to fix the issue asap.

Tester should focuses on the software product itself and gathers important information regarding what it does and doesn’t do. The process of gathering should include all the teams associated with the product. Talks to management, sales, architects, development, support and gather their expectations over the release.

Don’t you think that this is a big job all by itself and need lot of time. As i keep saying the Role of the Tester is to provide qualitative information on the product to the management for the better decisions. So the big challenge here is to provide accurate, comprehensive, and timely information about the product under development.


%d bloggers like this: