2017 OSHG: IMA Test WG: Problem analysis tests, objects and protocols: Example: TMFFail

Problem analysis tests, objects and protocols: 
Example: TMFFail

Sometimes a persistent bug/defect is noted and the source is not immediately obvious.  No one can investigate a problem unless it can be replicated.  This becomes a greater problem if the defect is intermittent.  In such cases, the IMA TestWG may be able to create a test object and test protocol to support investigation of the problem. 

TMFFail
The defect appears in two prims on the right
Click image to enlarge

Example problem 

• Texture multiple face fail (TMFFail)
• In an object with multiple faces, you can apply the same texture to all faces at once.
• This is done in the texture panel of the edit window.
• Sometimes the process fails on one or a few faces.
• The failure is intermittent, so not easy to replicate.
• This test object makes replication easier.  
• Thanks to Kayaker Magic for the initial design of this test object.
• And gives more confidence of failure to replicate when that is found.
• The  test object might later be incorporated into the QA test suite.

****************************

Another example

Texture, multiple face (TMFFail)

We have discussed this problem in the Kitely meetings.  Since I am forming an IMA Test Working Group, I am investigating this problem.  I made a test object that can reliably demonstrate  the effect.  With that object, I can investigate the conditions under which it occurs.  
-
This investigation is informal at this stage, meaning that I am not getting full details off the test results.  I am just seeking answers to the initial questions raised by the issue.
-
1. Is this problem limited to Kitely?  

• Definitely not.  We (the test WG) have replicated the effect on several other grids, running OS 0.8.x and OS 0.9.x.  

2. Does this effect appear in SL?  

• Apparently not. We have not been able to replicate it there.

3. Is this a viewer fault? 

• Apparently not,  We used FS32 in OpenSim and in SL.  
• We found the effect in OS, but not in SL.

4. Does the effect persist in the object across copies and after take and rez?

• Definitely yes.

5. Can we find any OpenSim grids where this problem does not appear?

• Yes.  The Encore Escape.    This grid s running:
• OpenSim 0.8.2.1-DWG16a-Post_Fixes       (Win/.NET)
• HG address:
• goto.theencoreescape.com:8002
• I cannot conclude that the problem does not appear ever.  
• I can conclude that I could not demonstrate it with my test object.
• With the number of trials that easily showed the problem on other OS grids.
• This version of OpenSimulator seems to be on Digiworldz also.
• Terry Ford may know how to fix the problem.  
• Perhaps he will contribute the solution back to the OpenSim community

6.  Other questions of possible interest: 

• Does this problem appear in Halcyon, Whitecore, or Arriba?
• Do we know enough about the problem to submit a mantis?

Test object (prims, not textures)

• make one torus, cut it to half-donut shape, hollow the half-donut.
• This makes 4 faces, all visible from the cut side.
• replicate to make 6 in a column, link.
• replicate linked column to make 12, link to make test object.

Test Protocol

• Edit test object
• In texture panel, click on texture display to get texture selections
• Using the OpenSim library, change the textures rapidly.
• Watch the cut side to see all the sides (outer, hole, two cut surfaces).
• All textures in the object should change.  
• If some faces do not get the new texture, that is a fail.
• If you see a fail, take a still picture and stop testing.
• You may need to work rapidly to get fails.
• Stop after the first fail, note the OS version and the viewer.
• Note anything else you think might be relevant to the problem.  
• Run at least 12 tries before concluding that it does not fail.
• If it does not fail, mention any conditions about your test that might be relevant.

IMA TestWG role in problem analysis

• Find or help develop a standard test to replicate the problem.
• Standard test:  Object or OAR with test protocol.
• Get enough tests done to indicate software at fault.
• Seek remedial action from appropriate source.
• Archive and curate the test materials and results.

IMA test planning

• Problem analysis tests, objects and protocols: Example: TMFFail
• Comparison suites for versions, modules, etc.
• Test WG: suites needed: Quality Assurance

Follow-up

BobC June 9, 2017 at 5:19 PM

Texture test run in IMA Halcyon Test Grid with no problems. Using Firestormx64 5.0.1 Halcyon 0.9.34. Textures changed as fast as I could click on them with all faces changing at the same visual instant. Halcyon uses a different process for texture processing than OpenSim does. It is much faster.
********
Terry Ford  June 15, 2017 at 6:24 PM

I have never specifically addressed this issue in our code (DigiWorldz and grids hosted by DigiWorldz), but I suspect either one of our other code "tweaks".. or a configuration setting is the reason our code seems unaffected.

I do not know of a specific "Fix" I have done which addressed this issue.

***************

Terry Ford  June 15, 2017 at 6:31 PM

Additionally, I'm wondering if you have tested for this bug on DigiWorldz, 3rd Rock Grid, Great Canadian Grid, Baller Nation, Sinful Grid, Origins Virtual World, and Genesis Metaverse?

I would be curious on your findings regarding each.

Of those listed, only Origins Virtual World is running 9.x code.. all others are running 8.x code.

If you test on Origins and the bug is not present, I would suspect a configuration setting, or Hardware/OS specific issue as all of our grids user Windows Server 2012 R2. Do you know if any of the other grids you've tested on were running Windows?

Reply:

Thanks for your suggestions. One of my volunteers did test it on Digiworldz and found that no fail was observed there. I have not gotten any results from the other grids and will look into some of them.  I have not been getting the info regarding Windows Server

I hear informally, however, that Ubit may have found a fix for OS 0.9. 

Clarification: A fix need not guarantee that it never happens. A fix is adequate if it reduces the frequency of failure enough so that it does not interfere with building by content creators