Regression in crypt32-cert winetest!

Here you can discuss ReactOS related topics.

Moderator: Moderator Team

Post Reply
Lone_Rifle
Test Team
Posts: 802
Joined: Thu Apr 03, 2008 2:17 pm
Contact:

Regression in crypt32-cert winetest!

Post by Lone_Rifle »

Using a self-written script to parse the buildbot logs...
Revision 38738:
http://reactos.agilityhoster.com/get.php?revision=38738
crypt32-cert: executed 366, 0 failures
http://reactos.agilityhoster.com/builds ... 2-cert.txt

Revision 38743:
http://reactos.agilityhoster.com/get.php?revision=38743
crypt32-cert: executed 356, 3 failures
http://reactos.agilityhoster.com/builds ... 2-cert.txt

Somebody get in touch with the devs please. Thanks.

Christoph_vW
Developer
Posts: 48
Joined: Sun Sep 18, 2005 11:58 pm
Location: Germany
Contact:

Re: Regression in crypt32-cert winetest!

Post by Christoph_vW »

looks pretty random:

r38745: cert: 361 tests executed (0 marked as todo, 2 failures), 0 skipped.
r38744: cert: 361 tests executed (0 marked as todo, 1 failure), 0 skipped.
r38743: cert: 356 tests executed (0 marked as todo, 3 failures), 0 skipped.
r38742: cert: 366 tests executed (0 marked as todo, 0 failures), 0 skipped.
r38741: cert: 361 tests executed (0 marked as todo, 2 failures), 0 skipped.

vicmarcal
Test Team
Posts: 2732
Joined: Mon Jul 07, 2008 12:35 pm

Re: Regression in crypt32-cert winetest!

Post by vicmarcal »

Ok Lone-Rifle...They have been notified.Thanks a lot.

BTW i love ur script...a nice way to compare results,finding regressions
Could you improve it making also a compare Diff?I mean you choose two revisions(or just comparing with previous revision) and show just the Tests that have changed,and how they have changed:if Reactos made a regression, if Reactos is passing/skipping more tests...So Devs can find them faster.

Lone_Rifle
Test Team
Posts: 802
Joined: Thu Apr 03, 2008 2:17 pm
Contact:

Re: Regression in crypt32-cert winetest!

Post by Lone_Rifle »

I'm working on it in my head. =)
Just bear in mind that this is just a stopgap measure cum public information tool so that people can have a clearer idea of what to look for in our source tree.

Another thing i missed: the scripts seem to pick up that the advapi32-service test now has 3 more failures from 38738 to 38743.
http://reactos.agilityhoster.com/builds ... ervice.txt
http://reactos.agilityhoster.com/builds ... ervice.txt

vicmarcal
Test Team
Posts: 2732
Joined: Mon Jul 07, 2008 12:35 pm

Re: Regression in crypt32-cert winetest!

Post by vicmarcal »

uhmm...maybe cert test have discovered a random bug.

Lone Rifle, a question and a enhancement:

1)How long does the full test take?

Enhancement:
Could you make the test TWICE each revision?Then script compare both Logs (from same revision) and create a Final Log where is checked as "Random Failures" failures that seems to be Random(maybe You can add a new COL). So then Devs can look Test code(maybe it´s wrong) or search why ReactOS fails randomly.

User avatar
Black_Fox
Posts: 1584
Joined: Fri Feb 15, 2008 9:44 pm
Location: Czechia

Re: Regression in crypt32-cert winetest!

Post by Black_Fox »

Lone_Rifle parses what is already available at BuildBot (eg. here), no additional testing.

Post Reply

Who is online

Users browsing this forum: Google [Bot], Semrush [Bot], Yandex [Bot] and 1 guest