How to be a Good Bug Buster (beta-tester)

What? Why?

If you think that searching for bugs is just run an application to see if it launches, then you need to continue reading this or work on something else, like translations.

Is very helpful to have some programming knowledges to do beta-testing, because on that way you know what common things could happen

Well, first of all, trying to just launch the application to see if does not segfaults doesn't give anything useful, in short this is exactly what the developer does in every modification of the code, the task of a beta-tester is not to see if it works but to see if it can fail, in short, it is like a challenge trying to find any possible problem, and yes, it is something good if you find any.

There is a good numbers of ways to catch bugs, so let me show you some of the ways that I already know:

Environment

Not everybody works in the same environment or like the same options, so if the developer works with the feature of application A, but he never uses B, it is very possible that there will be more bugs in "B" than in "A", in the same way, by trying not very interesting options you will have the same results

Try to imaginate how many different environments and different ways to work can exist, for example if you are beta-testing enlightenemnt 17, and you want to see if some window minimizes correctly, think about all the possibilities, you can do it by using the mouse to click the minimize button, or you can also press ctrl + alt + i on the keyboard to do the same thing, also think in other things related to minimizing an application, for example engage or taskbar can trigger the minimize too

Unimaginable Possibilities

Are you creative ? you need to be... think about it, in the previous example we have learnt different ways to trigger the same thing, but all the possibilities are not listed here, you need to be creative (or at least get familiarized with the application/code) to imagine how much different things can happen, for example:

• can the window be minimized when is in maximized mode ?
• if another window is on top of another one, we can still minimize it ?
• what happens if I try to minimize during an e17 restart ?
• which window is minimized by default if I switch to a different desktop and don't manually focus in any ?
• what happens if the minimizer module is not loaded ?

The previous list is just an example and you can find them useful or not to test, it all depends on you, but basically the important thing is to try everything that comes to your head that makes sense.

Combinations

You can more easily find bugs if you try to combine things, for example:

• what happens if you run the application multiple times ?
  • is the performance affected by running it multiple times ?
• try A while you are doing B
• do the same tests with different settings
• combine parameters/features at the same time

Frequent Problems in applications

This is more like a list of common noobish programming issues, so if you find them, you can spank the author of the application ;)

• Your system is multiuser, there's a lot of applications that uses /tmp to store temporary data (like /tmp/app1), but if another user runs the same application, it will use the same location, and this can result in really big, annoying and even hard to find problems, the most common ones are permission denied reports
• Multiple instances: sometimes an application is not ready to be run multiple times, this needs to be correctly managed by the programmer in order to not conflict with different instances of the same application, if the application is made to only be able to run one instance, then it needs to avoid another run with a message like application is already running

Other different types of bugs

Pebkac

(Problem Exist Between Keyboard And Chair): after to spank the user of your application because he/she is stupid, you must give yourself some spankies too, because pebkac's are really bugs that you need to fix... yes, you, the developer :), that's called an usability issue, you may think that the users are stupid but this won't change anything, what can change is your application, make it more evident, lazy dev!

Wording

A grammar error is also an error to report, if you find some wrong messages like for example the ones written by the illiterate Thanatermesis, you must report it too... Update: Since version Topaz, the translation tool of Elive also includes the possibility to translate to english, this means, from tarzanic (application orginal) to english!

The wording beta-testing doesn't finish here, an interface must use the optimal messages, in short this means:

• avoid flood of boring information (users don't read, realize it)
• clear and direct messages, without possible different interpretations
• short messages
• easy to understand, but not for dumbs changing the logic and sense

User Experience (UX)

This is an entire science, so it requires some knowledge about what is and what is not a good interface, lots of knowledge exist about it... but by other side, you can still check these things, because you are a logic and coherent person, the interfaces of the applications needs to be coherent and logic, easy to use, handy, userfriendly, etc... so if you think that the interface can be better made in a different way, just report it with the details of how it should be made.

This involves:

• User interface Design (UI)
• Optimal Positioning of elements (buttons, text, etc)
• Feelings that gives the interface, the use of it, the actions, etc
• Usability to use
• Information architecture (IA)
• Handy application, obtaining efficient and desired results
• Helping tips (way's to help the user if some element is difficult)
• Avoiding overload of elements/text/images/tips/messages/etc
• Interaction design (IxD)
• Psychology

Betatest yourself

It is very common that we report not correct bugs, this happens when the cause of the problem is not what you think but just a side effect of another thing, a similar problem occurs when you report that the cause is A but in fact is in another place. Just make sure the real cause of the problem before to report it.

Remember

It is very important to report the information to the developers, problems doesn't solve themself, so please do it, you are the real end beneficiary of the correctly working application, not the developer

Think as if you were the developer, first of all you need to be able to reproduce the problem so you need to give the developer the required information to reproduce it (do it yourself, and write the steps). Finally, you need to include all the relevant information that the developer may need, make his work more easy and you will have more possibilities to have it fixed!

Before you report any bug, try to have someone else reproduce the bug, this can confirm it and also verify the correct reproduction of it, you can ask any volonteer in the IRC chat channel

Motivational end sentence: Girl's doesn't really like muscles, they likes inteligent guys, smart beta-testers that catches lots of bugs are very intelligent guys.

You don't belive me ? ​see it yourself with your eyes, really... ​it is that way