How to be a Good Bug Buster (beta-tester)

What? Why?

If you think that searching for bugs is just run an application to see if it launches, then you need to read this howto

If you have programming knowledge it can be very helpful so on that way you know what common things could happen

First of all, trying to just launch the application to see if does not segfaults doesn't give anything useful, it is like a challenge trying to find how it can fail.

But there's some techniques you can use to find these hidden bugs, so let me show you some tips:

Environment

Not everybody works in the same environment or using the same options, so if the developer works with the feature of application A, but he never uses B, it is very possible that there will be more bugs in "B" than in "A", in the same way, by trying not very interesting options you will have the same results

Try to imagine how many different environments and different ways to work can exist, for example if you are beta-testing Enlightenment 17, and you want to see if some window minimizes correctly, think about all the possibilities, you can do it by using the mouse to click the minimize button, or you can also press ctrl + alt + i on the keyboard to do the same thing, also think in other things related to minimizing an application, for example engage or taskbar can trigger the minimization too.

Unimaginable Possibilities

Are you creative ? you need to be... think about it, in the previous example we have seen different ways to trigger the same thing, but all the possibilities are not listed here, you need to be creative (or at least get familiarized with the application/code) to imagine how much different things can happen, for example:

• can the window be minimized when is in maximized mode ?
• if another window is on top of another one, we can still minimize it ?
• what happens if I try to minimize during an e17 restart ?
• which window is minimized by default if I switch to a different desktop and don't manually focus in any ?
• what happens if the minimizer module is not loaded ?

The previous list is just an example and you can find them useful or not to test, it all depends on you, but basically the important thing is to try everything that comes to your head that makes sense.

Expect the unexpectable

Biggest Headaches

If you have the biggest headaches trying to catch a bug and you still don't found it, may be because you are not looking at the correct place. There's a few examples:

• Warning with async, threads, and parallel tasks: If you have things that runs in paralel, nothing ensures you that A would happen before B even if in the code its declared on this way. For example: You have a code that generates a temporal html file with statistics, then you open it with a browser in background to visualize it, then you copy it to a backup directory and finally since this is a temporal file you simply delete it, on this example you could experience that the browser fails, why this could happen? I used the example of a browser because is a bloated ton of crap and so it takes some time to open, in the meantime the script has continued and removed the file, when the browser is ready to print it the file is gone.

Combinations

You can more easily find bugs if you try to combine things, for example:

• what happens if you run the application multiple times ?
  • is the performance affected by running it multiple times ?
• try A while you are doing B
• do the same tests with different settings
• combine parameters/features at the same time

Frequent Problems in applications

This is more like a list of common noobish programming issues, so if you find them, you can spank the author of the application ;)

• Your system is multiuser, there's a lot of applications that uses /tmp to store temporary data (like /tmp/app1), but if another user runs the same application, it will use the same location, and this can result in really big, annoying and even hard to find problems, the most common ones are permission denied reports
• Multiple instances: sometimes an application is not ready to be run multiple times, this needs to be correctly managed by the programmer in order to not conflict with different instances of the same application, if the application is made to only be able to run one instance, then it needs to avoid another run with a message like application is already running

Betatest yourself

It is very common that we report not correct bugs, this happens when the cause of the problem is not what you think but just a side effect of another thing, a similar problem occurs when you report that the cause is A but in fact is in another place. Just make sure the real cause of the problem before to report it.

Other different types of bugs

Pebkac

(Problem Exist Between Keyboard And Chair): after to spank the user of your application because he/she is stupid, you must give yourself some spankies too, because pebkac's are really bugs that you need to fix... yes, you, the developer :), that's called an usability issue, you may think that the users are stupid but this won't change anything, what can change is your application, make it more evident, lazy dev!

Wording

A grammar error is also an error to report, if you find some wrong messages like for example the ones written by the illiterate Thanatermesis, you must report it too... Update: Since version Topaz, the translation tool of Elive also includes the possibility to translate to english, this means, from tarzanic (application orginal) to english!

The wording beta-testing doesn't finish here, an interface must use the optimal messages, in short this means:

• avoid flood of boring information (users don't read, realize it)
• clear and direct messages, without possible different interpretations
• short messages
• easy to understand, but not for dumbs changing the logic and sense

User Experience (UX)

This is an entire science, so it requires some knowledge about what is and what is not a good interface, lots of knowledge exist about it... but by other side, you can still check these things, because you are a logic and coherent person, the interfaces of the applications needs to be coherent and logic, easy to use, handy, userfriendly, etc... so if you think that the interface can be better made in a different way, just report it with the details of how it should be made.

This involves:

• User interface Design (UI)
• Optimal Positioning of elements (buttons, text, etc)
• Feelings that gives the interface, the use of it, the actions, etc
• Usability to use
• Information architecture (IA)
• Handy application, obtaining efficient and desired results
• Helping tips (way's to help the user if some element is difficult)
• Avoiding overload of elements/text/images/tips/messages/etc
• Interaction design (IxD)
• Psychology

Remember

It is very important to report the information to the developers, problems doesn't solve themself, so please do it, you are the real end beneficiary of the correctly working application, not the developer

Think as if you were the developer, first of all you need to be able to reproduce the problem so you need to give the developer the required information to reproduce it (do it yourself, and write the steps). Finally, you need to include all the relevant information that the developer may need, make his work more easy and you will have more possibilities to have it fixed!

Before you report any bug, try to have someone else reproduce the bug, this can confirm it and also verify the correct reproduction of it, you can ask any volonteer in the IRC chat channel

Motivational end sentence: Girl's doesn't really like muscles, they likes inteligent guys, smart beta-testers that catches lots of bugs are very intelligent guys.

You don't belive me ? ​see it yourself with your eyes, really... ​it is that way