| 3 | | These steps explain how to correctly report bugs and how to be a proper beta-tester. Reporting the correct information, though not too much, is more important. Quality, not quantity. The concept of a beta tester is not to try an application to see if works but to make it not work, and report it. |
| 4 | | |
| 5 | | == Essential points to remember when beta-testing == |
| 6 | | |
| 7 | | 1. Check if the program works well |
| 8 | | 2. Search for bugs or visual defects |
| 9 | | 3. Test all the different parts of the application to see if the program works well entirely and all its options available |
| 10 | | 4. If you are on this step and everything before works correctly, try to do all the possible combinations of the parts of the application to find a possible bug |
| 11 | | 5. Use your brain, so try using the application under many different circumstances to see if it works with other applications and the distro as a whole. For example, configure the network with different interfaces (eth0, eth1, ath0, wifi0, ra0...), also test what happens if you configure the network and disconnect the cable, etc, be ingenious |
| 12 | | 6. Think how the program works. The work of a beta-tester is not to show how well the program works. It is the complete opposite. Your mission is to try to crash the application, to find a bug. Just as a cracker looks for a bug to enter a system without privileges, you need to be more intelligent than the machine. You need to understand how the machine and the program works and how the program communicates with the machine. You should try to confuse the program within limits. You need to beat the program. Only then... are you really satisfied. |
| 13 | | 7. Finally, and the most important one... try to report all the possible information (only useful) in order that the developer could fix the problem in the most easy and fast way... if you fix it, you can include a patch (or explanation) in the report, they are more than welcome! |
| | 3 | [[Image(http://main.elivecd.org/images/misc/tux-chemical.png)]] |
| 16 | | '''Note:''' ''If it is not possible to test something, inform others so that they may do the test.'' |
| | 6 | = How to be a Good Bug Buster (beta-tester) = |
| | 7 | |
| | 8 | === What? Why? === |
| | 9 | If you think that searching for bugs is just run the application to see if launches, then you need to continue reading this or work in a different thing, like translations. |
| | 10 | |
| | 11 | Is very helpful to have some programming knowledges for do some beta-testing, because on that way you know ''what common things could happen'' |
| | 12 | |
| | 13 | Well, first of all, trying to just launch the application to see if not segfaults doesn't give anything useful, in short this is what the developer exactly does in every modification of the code, the task of a beta-tester is not to see if works but to ''to see if can fail'', in short, is like a challenge trying to found any possible problem, and yes, it is something '''good''' if you found any. |
| | 14 | |
| | 15 | There is a good numbers of tips by being able to catch them, so let me show you some of the tips that I already know: |
| | 16 | |
| | 17 | === Environment === |
| | 18 | Not everybody works in the same environment or like the '''same options''', so if the developer works with the feature of the application ''A'', but he never uses ''B'', is very possible that there will be more bugs in B than in A, in the same way, by trying ''not very interesting options'' you will have the same results |
| | 19 | |
| | 20 | Try to imaginate how much different environments and different ways of work can exist, for example if you are beta-testing enlightenemnt 17, and you want to see if some window minimizes correctly, think about all the possibilities, for example you can do it by clicking with the mouse in the minimizer button, or you can also press ''ctrl + alt + i'' which is the keyboard-mode to do the same thing, also think in other things related with the minimizer, for example engage or task bars can trigger the minimizer too |
| | 21 | |
| | 22 | === Unimaginable Possibilities === |
| | 23 | Are you creative ? you need to... think on that, in the previous example we have tough in different ways to trigger the same thing, but all the possibilities not stick just here, you need to be creative (or at least familiarized with the application/code) to imagine how much different things can happen, for example: |
| | 24 | * can the window be minimized when is in maximized mode ? |
| | 25 | * if another window is on top of our one, we can still minimize it ? |
| | 26 | * what happens if i try to minimize during an e17 restart ? |
| | 27 | * which window is minimized by default if i switch to a different desktop and don't manually focus in any ? |
| | 28 | * what happens if the minimizer module is not loaded ? |
| | 29 | |
| | 30 | The previous list is just an example and you can found them useful or not to test, it all depends of you, but basically the important thing is to try everything that comes to your head that can have sense. |
| | 31 | |
| | 32 | === Combinations === |
| | 33 | You can more easly found bugs if you try to combine things, for example: |
| | 34 | * what happens if you run the application multiple times ? |
| | 35 | * is the performance affected by running it multiple times ? |
| | 36 | * try A while you are doing B |
| | 37 | * do the same tests with different settings set |
| | 38 | * combine parameters/features at the same time |
| | 39 | |
| | 40 | === Frequent Problems in applications === |
| | 41 | This is more like a list of ''common noobish programming issues'', so if you found them, you can spank the author of the application ;) |
| | 42 | |
| | 43 | * Your system is multiuser, there's a lot of applications that uses /tmp for store some temporal data (like /tmp/app1), but if another user runs the same application, it will use the same location, and this can result in really big, annoying and even hard to found problems, the most common ones are ''permission denied'' reports |
| | 44 | * Multiple instances: sometimes an application is not ready to be run multiple times, this needs to be correctly managed by the programmer in order to not conflict with different instances of the same application, if the application is made to only be able to run one instance, then needs to avoid another run with a message like ''application is already running'' |
| | 45 | |
| | 46 | === Other different types of bugs === |
| | 47 | ==== Pebkac ==== |
| | 48 | ''(Problem Exist Between Keyboard And Chair)'': after to spank the user of your application because is stupid, you must give you some spankies too, because pebkac's are '''really''' bugs that '''you''' need to fix... yes, you, the developer :), that's called an ''usability issue'', you may think that the users are stupid but this won't change anything, what can change is your application, make it more evident, lazy dev! |
| | 49 | |
| | 50 | ==== Wording ==== |
| | 51 | A grammal error is also an error to report, if you found some wrong messages like for example the ones writed by the illiterate of ''Thanatermesis'', you must report it too... '''Update''': Since version Topaz, the translation tool of Elive also includes the possibility to translate to english, this means, from tarzanic (application orginal) to english! |
| | 52 | |
| | 53 | The wording beta-testing doesn't finish here, an interface must use the optimal messages, in short this means: |
| | 54 | * avoid flood of boring information (users don't read, realize it) |
| | 55 | * clear and direct messages, without possible different interpretations |
| | 56 | * short messages |
| | 57 | * easy to understand, but not for dumbs changing the logic and sense |
| | 58 | |
| | 59 | ==== User Experience (UX) ==== |
| | 60 | This is an entire science, so it requires some knowledge about what is and what is not a good interface, lots of knowledge exist about it... but by other side, you can still check these things, because you are a '''logic and coherent''' person, the interfaces of the applications needs to be coherent and logic, easy to use, handy, userfriendly, etc... so if you think that the interface can be better made in a different way, just report it with the details of how must be made. |
| | 61 | |
| | 62 | This involves: |
| | 63 | * User interface Design (UI) |
| | 64 | * Optimal Positioning of elements (buttons, text, etc) |
| | 65 | * Feelings that gives the interface, the use of it, the actions, etc |
| | 66 | * Usability to use |
| | 67 | * Information architecture (IA) |
| | 68 | * Handy application, obtaining efficient and desired results |
| | 69 | * Helping tips (way's to help the user if some element is difficult) |
| | 70 | * Avoiding overload of elements/text/images/tips/messages/etc |
| | 71 | * Interaction design (IxD) |
| | 72 | * Psychology |
| | 73 | |
| | 74 | [[Image(http://uxdesign.com/assets/ux-defined/user-experience-design-diagram.jpg)]] |
| | 75 | |
| | 76 | |
| | 77 | === Remember === |
| | 78 | It is very important to report the information to the developers, problems doesn't solve by themself, so please do it, you are the real end beneficiary of the correctly working application, not the developer |
| | 79 | |
| | 80 | Think like you was the developer, first of all you need to be able to '''reproduce''' the problem so you need to give him the required information for reproduce it (do it yourself, and write the steps). Finally, you need to include all the relevant information that the developer may need, make his work more easy and you will have more possibilities to have it fixed! |
| | 81 | |
| | 82 | Before to report any bug, try to somebody else can be reproduce the bug, this can confirm it and also verify the correct reproduction of it, you can ask any volonteer in the IRC chat channel |
| | 83 | |
| | 84 | Motivational sentence: Girl's doesn't really like muscles, they likes inteligent guys, smart beta-testers that catches lots of bugs are very intelligent guys. |
| | 85 | |
