Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
briar
briar
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 693
    • Issues 693
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge Requests 16
    • Merge Requests 16
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
  • Operations
    • Operations
    • Incidents
    • Environments
  • Analytics
    • Analytics
    • CI / CD
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • briar
  • briarbriar
  • Issues
  • #1560

Closed
Open
Opened May 02, 2019 by iwakeh@iwakeh

test results of ForumSharingIntegrationTest depend on test execution order

I came across this by chance, but the changes (diff below) make the problem reproducible.

Steps to reproduce:

Running testInviteeLeavesAfterFinished before testSharerLeavesAfterFinished will lead to the following test failure:

java.lang.AssertionError: expected:<1> but was:<0>
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failNotEquals(Assert.java:834)
	at org.junit.Assert.assertEquals(Assert.java:645)
	at org.junit.Assert.assertEquals(Assert.java:631)
	at org.briarproject.briar.sharing.ForumSharingIntegrationTest.testSharerLeavesBeforeResponse(ForumSharingIntegrationTest.java:396)
...

(just one example, there are more test orders to reproduce this)

code to reproduce:

index 987a78d20..2b042455f 100644
--- a/briar-core/src/test/java/org/briarproject/briar/sharing/ForumSharingIntegrationTest.java
+++ b/briar-core/src/test/java/org/briarproject/briar/sharing/ForumSharingIntegrationTest.java
@@ -43,6 +43,8 @@ import static org.junit.Assert.assertFalse;
 import static org.junit.Assert.assertNull;
 import static org.junit.Assert.assertTrue;
 
+import org.junit.FixMethodOrder;
+import org.junit.runners.MethodSorters;@FixMethodOrder(MethodSorters.NAME_ASCENDING)
 public class ForumSharingIntegrationTest
                extends BriarIntegrationTest<BriarIntegrationTestComponent> {
 
@@ -235,7 +237,7 @@ public class ForumSharingIntegrationTest
        }
 
        @Test
-       public void testInviteeLeavesAfterFinished() throws Exception {
+       public void test1InviteeLeavesAfterFinished() throws Exception {
                // initialize and let invitee accept all requests
                listenToEvents(true);
 
@@ -292,7 +294,7 @@ public class ForumSharingIntegrationTest
        }
 
        @Test
-       public void testSharerLeavesAfterFinished() throws Exception {
+       public void test2SharerLeavesAfterFinished() throws Exception {
                // initialize and let invitee accept all requests
                listenToEvents(true);

Suggestion: prepare each test in ForumSharingIntegrationTest without relying on any state created by other tests.

Assignee
Assign to
Android 1.1
Milestone
Android 1.1
Assign milestone
Time tracking
None
Due date
None
Reference: briar/briar#1560