An exception in OnAssemblyStart should stop the execution of the specs in the assembly

[ulrichb]

... to avoid consequential errors in the specs.

The following code sample demonstrates the issue: although the first exception is displayed in the R# and console runner, the execution continues and throws a NullReferenceException within When_using_the_context_value as a consequence of the exception in the AssemblyContext. The current behavior is more annoying if the relationship between the assembly context and the specs is more indirect (like creating a directory in the assembly context and use that in the specs or setting up some test data in a database in the assembly context) because then it's harder to detect that the specs just fail because of an error in the assembly context.

Further, NUnit's [SetUpFixture] and xUnit.net's IUseFixture<> also stop the execution of the concerned tests.

public class AssemblyContext : IAssemblyContext
{
    public static SomeContext SomeContext;

    public void OnAssemblyStart()
    {
        // SomeContext will stay null because the c'tor throws:
        SomeContext = new SomeContext();
    }

    public void OnAssemblyComplete() { }
}

public class SomeContext
{
    public SomeContext()
    {
        throw new Exception();
    }

    public object Value { get { return new object(); } }
}

class SomeSpec
{
    class When_using_the_context_value
    {
        Because of = () =>
        {
            // This will throw a NullReferenceException because 
            // AssemblyContext.SomeContext == null:
            Console.WriteLine(AssemblyContext.SomeContext.Value);
        };

        It happens_something = () => { };
    }
}

[danielmarbach]

I just ran several tests on this. An exception with the full stack trace is raised inside resharper. See for example:

After you click OK all the specs are executed nonetheless and the mentioned NullReferenceException is raised. This is exactly the same behavior as when you have a specification which has an exception in the Establish clause.

In order to fix this we need to make IAssemblyContext a first class citizen in the framework which is a major task. As mentioned because the behavior is the same as in the establish clause I consider this works as intented. I leave it in the backlog for future reference.

[danielmarbach]

I'll be more than happy to review pull requests if this is a major show stopper for you.

[ulrichb]

This is exactly the same behavior as when you have a specification which has an exception in the Establish clause.

In the following example, when executing it in the R# or console runner, you will see the Exception thrown in the Establish step and no consequential NullReferenceException because the execution of the scenario stops immediately (same behavior, if the Establish is moved into When_using_the_context_value).

public class SomeContext
{
    public SomeContext()
    {
        throw new Exception();
    }

    public object Value { get { return new object(); } }
}

class SomeSpec
{
    static SomeContext SomeContext;

    Establish ctx = () =>
    {
        SomeContext = new SomeContext();
    };

    class When_using_the_context_value
    {
        Because of = () =>
        {
            Console.WriteLine(SomeContext.Value);
        };

        It happens_something = () => { };
    }
}

[drauch]

Any updated on this?