[ANNOUNCE] TestSetExtensions.jl

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[ANNOUNCE] TestSetExtensions.jl

Spencer Russell-4
Hey All,

I just registered the TestSetExtensions package, which collects some extensions and convenience utilities to maximize your testing enjoyment. It builds on the new Base.Test infrastructure in Julia v0.5 (also available in v0.4 with the BaseTestNext package). It's designed so that you shouldn't need to modify your tests at all if you're already using @testset and @test - all the interactions with this package happen at the top-level of your tests.


Right now is supports two features:

  1. Printing out green dots as your tests run so you can see them progressing
  2. Collecting your test files and running them automatically so you don’t have to manually add them to `runtests.jl`, and also so you can easily specify a sub-set of your tests to run from the command line

I mostly wrote it to encapsulate the things I want for my tests, but I’m happy to take PRs for new features.

Please kick the tires and let me know if it works (or doesn’t work) for you!

ssfrr
Reply | Threaded
Open this post in threaded view
|

Re: [ANNOUNCE] TestSetExtensions.jl

pdobacz
Hey Spencer,

I like this! 

I have recently started a testing-related package as well, maybe you've noticed (https://github.com/pdobacz/PyTest.jl). It seems that both packages could work together and complement quite seamlessly, so I'll definitely give yours a try. 

BTW, do you think there is much demand for Julia testing tools?
Reply | Threaded
Open this post in threaded view
|

Re: [ANNOUNCE] TestSetExtensions.jl

Florian Oswald
In reply to this post by Spencer Russell-4
Good stuff! I like the idea of extending Base.Test

anyway you could easily integrate printing messages after failed tests?

https://groups.google.com/d/msg/julia-users/_cZ8y_-JAVA/8EcTWdxcAQAJ

cheers

On Thursday, 27 October 2016 17:27:38 UTC+2, Spencer Russell wrote:
Hey All,

I just registered the TestSetExtensions package, which collects some extensions and convenience utilities to maximize your testing enjoyment. It builds on the new Base.Test infrastructure in Julia v0.5 (also available in v0.4 with the BaseTestNext package). It's designed so that you shouldn't need to modify your tests at all if you're already using @testset and @test - all the interactions with this package happen at the top-level of your tests.

<a href="https://github.com/ssfrr/TestSetExtensions.jl" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fssfrr%2FTestSetExtensions.jl\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEDZmgfNgSOfyRYnYtVBlqv6se-MA&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fssfrr%2FTestSetExtensions.jl\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEDZmgfNgSOfyRYnYtVBlqv6se-MA&#39;;return true;">https://github.com/ssfrr/TestSetExtensions.jl

Right now is supports two features:

  1. Printing out green dots as your tests run so you can see them progressing
  2. Collecting your test files and running them automatically so you don’t have to manually add them to `runtests.jl`, and also so you can easily specify a sub-set of your tests to run from the command line

I mostly wrote it to encapsulate the things I want for my tests, but I’m happy to take PRs for new features.

Please kick the tires and let me know if it works (or doesn’t work) for you!

ssfrr
Reply | Threaded
Open this post in threaded view
|

Re: [ANNOUNCE] TestSetExtensions.jl

Spencer Russell-4
Thanks for the feedback! Part of Iain’s original design for the new Base.Test was explicitly to support extension via `AbstractTestSet` subtypes, so hopefully we’ll start see more use of the feature.

I’m not sure I understand your question about printing messages after failed tests - Base.Test prints an error and backtrace by default for each failing test. What’s the behavior you’re looking for?

-s


On Oct 29, 2016, at 8:46 AM, Florian Oswald <[hidden email]> wrote:

Good stuff! I like the idea of extending Base.Test

anyway you could easily integrate printing messages after failed tests?


cheers

On Thursday, 27 October 2016 17:27:38 UTC+2, Spencer Russell wrote:
Hey All,

I just registered the TestSetExtensions package, which collects some extensions and convenience utilities to maximize your testing enjoyment. It builds on the new Base.Test infrastructure in Julia v0.5 (also available in v0.4 with the BaseTestNext package). It's designed so that you shouldn't need to modify your tests at all if you're already using @testset and @test - all the interactions with this package happen at the top-level of your tests.

<a href="https://github.com/ssfrr/TestSetExtensions.jl" target="_blank" rel="nofollow" onmousedown="this.href='https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fssfrr%2FTestSetExtensions.jl\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEDZmgfNgSOfyRYnYtVBlqv6se-MA';return true;" onclick="this.href='https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fssfrr%2FTestSetExtensions.jl\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEDZmgfNgSOfyRYnYtVBlqv6se-MA';return true;" class="">https://github.com/ssfrr/TestSetExtensions.jl

Right now is supports two features:

  1. Printing out green dots as your tests run so you can see them progressing
  2. Collecting your test files and running them automatically so you don’t have to manually add them to `runtests.jl`, and also so you can easily specify a sub-set of your tests to run from the command line

I mostly wrote it to encapsulate the things I want for my tests, but I’m happy to take PRs for new features.

Please kick the tires and let me know if it works (or doesn’t work) for you!

ssfrr

Reply | Threaded
Open this post in threaded view
|

Re: [ANNOUNCE] TestSetExtensions.jl

Spencer Russell-4
In reply to this post by pdobacz
I think in general the culture in the Julia community is very pro-testing, which I really appreciate. I saw your post recently about PyTest, but I’m generally pretty happy with the built-in `@testset` / `@test` system, and just wanted some lightweight convenience functionality that wouldn’t require people to restructure their existing tests.

-s


On Oct 29, 2016, at 4:30 AM, [hidden email] wrote:

Hey Spencer,

I like this! 

I have recently started a testing-related package as well, maybe you've noticed (https://github.com/pdobacz/PyTest.jl). It seems that both packages could work together and complement quite seamlessly, so I'll definitely give yours a try. 

BTW, do you think there is much demand for Julia testing tools?

mmh
Reply | Threaded
Open this post in threaded view
|

Re: [ANNOUNCE] TestSetExtensions.jl

mmh
Will these improvements make into Base.Test ?

On Monday, October 31, 2016 at 12:53:37 AM UTC-4, Spencer Russell wrote:
I think in general the culture in the Julia community is very pro-testing, which I really appreciate. I saw your post recently about PyTest, but I’m generally pretty happy with the built-in `@testset` / `@test` system, and just wanted some lightweight convenience functionality that wouldn’t require people to restructure their existing tests.

-s


On Oct 29, 2016, at 4:30 AM, <a href="javascript:" target="_blank" gdf-obfuscated-mailto="urAGiB8pAgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">pdo...@... wrote:

Hey Spencer,

I like this! 

I have recently started a testing-related package as well, maybe you've noticed (<a href="https://github.com/pdobacz/PyTest.jl" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fpdobacz%2FPyTest.jl\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEw-bH-i87k3iw9JmhH_Rlr2qebRw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fpdobacz%2FPyTest.jl\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEw-bH-i87k3iw9JmhH_Rlr2qebRw&#39;;return true;">https://github.com/pdobacz/PyTest.jl). It seems that both packages could work together and complement quite seamlessly, so I'll definitely give yours a try. 

BTW, do you think there is much demand for Julia testing tools?

Reply | Threaded
Open this post in threaded view
|

Re: [ANNOUNCE] TestSetExtensions.jl

Spencer Russell-4
I'm not planning on merging this functionality into Base.Test. I think it's nice to keep Base.Test pretty minimal but extensible, and then have extra functionality provided via packages so they're easier for the community to iterate on and contribute to.

-s


On Mon, Oct 31, 2016, at 02:17 AM, mmus wrote:
Will these improvements make into Base.Test ?

On Monday, October 31, 2016 at 12:53:37 AM UTC-4, Spencer Russell wrote:
I think in general the culture in the Julia community is very pro-testing, which I really appreciate. I saw your post recently about PyTest, but I’m generally pretty happy with the built-in `@testset` / `@test` system, and just wanted some lightweight convenience functionality that wouldn’t require people to restructure their existing tests.

-s


On Oct 29, 2016, at 4:30 AM, [hidden email] wrote:

Hey Spencer,

I like this! 

I have recently started a testing-related package as well, maybe you've noticed (https://github.com/pdobacz/PyTest.jl). It seems that both packages could work together and complement quite seamlessly, so I'll definitely give yours a try. 

BTW, do you think there is much demand for Julia testing tools?

Reply | Threaded
Open this post in threaded view
|

Re: [ANNOUNCE] TestSetExtensions.jl

Iain Dunning
So happy you made this!

On Monday, October 31, 2016 at 4:25:47 PM UTC, Spencer Russell wrote:
I'm not planning on merging this functionality into Base.Test. I think it's nice to keep Base.Test pretty minimal but extensible, and then have extra functionality provided via packages so they're easier for the community to iterate on and contribute to.

-s


On Mon, Oct 31, 2016, at 02:17 AM, mmus wrote:
Will these improvements make into Base.Test ?

On Monday, October 31, 2016 at 12:53:37 AM UTC-4, Spencer Russell wrote:
I think in general the culture in the Julia community is very pro-testing, which I really appreciate. I saw your post recently about PyTest, but I’m generally pretty happy with the built-in `@testset` / `@test` system, and just wanted some lightweight convenience functionality that wouldn’t require people to restructure their existing tests.

-s


On Oct 29, 2016, at 4:30 AM, [hidden email] wrote:

Hey Spencer,

I like this! 

I have recently started a testing-related package as well, maybe you've noticed (<a href="https://github.com/pdobacz/PyTest.jl" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fpdobacz%2FPyTest.jl\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEw-bH-i87k3iw9JmhH_Rlr2qebRw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fpdobacz%2FPyTest.jl\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEw-bH-i87k3iw9JmhH_Rlr2qebRw&#39;;return true;">https://github.com/pdobacz/PyTest.jl). It seems that both packages could work together and complement quite seamlessly, so I'll definitely give yours a try. 

BTW, do you think there is much demand for Julia testing tools?

mmh
Reply | Threaded
Open this post in threaded view
|

Re: [ANNOUNCE] TestSetExtensions.jl

mmh
In reply to this post by Spencer Russell-4
My biggest problem with Base.Test is that you can't individually select which  testsets you want to run
i.e. Pkg.test("MyModule","Set1")
which makes testing really annoying.

This package addresses this issue (albeit a little differently) 

I do believe having a more featured Base.Test with a few more batteries included is an overall boon.


On Monday, October 31, 2016 at 12:25:47 PM UTC-4, Spencer Russell wrote:
I'm not planning on merging this functionality into Base.Test. I think it's nice to keep Base.Test pretty minimal but extensible, and then have extra functionality provided via packages so they're easier for the community to iterate on and contribute to.

-s


On Mon, Oct 31, 2016, at 02:17 AM, mmus wrote:
Will these improvements make into Base.Test ?

On Monday, October 31, 2016 at 12:53:37 AM UTC-4, Spencer Russell wrote:
I think in general the culture in the Julia community is very pro-testing, which I really appreciate. I saw your post recently about PyTest, but I’m generally pretty happy with the built-in `@testset` / `@test` system, and just wanted some lightweight convenience functionality that wouldn’t require people to restructure their existing tests.

-s


On Oct 29, 2016, at 4:30 AM, [hidden email] wrote:

Hey Spencer,

I like this! 

I have recently started a testing-related package as well, maybe you've noticed (<a href="https://github.com/pdobacz/PyTest.jl" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fpdobacz%2FPyTest.jl\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEw-bH-i87k3iw9JmhH_Rlr2qebRw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fpdobacz%2FPyTest.jl\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEw-bH-i87k3iw9JmhH_Rlr2qebRw&#39;;return true;">https://github.com/pdobacz/PyTest.jl). It seems that both packages could work together and complement quite seamlessly, so I'll definitely give yours a try. 

BTW, do you think there is much demand for Julia testing tools?

Reply | Threaded
Open this post in threaded view
|

Re: [ANNOUNCE] TestSetExtensions.jl

Spencer Russell-4
Normally I run my tests from the shell, so I focused on that use-case. I do think a bit that it would be cool to have a way to run the tests selectively from the julia REPL, but haven't put much time into thinking about the best way. Feel free to file an issue with some suggestions (or even better a PR) ;)

Then from a user's perspective it might be as easy as:

```
julia> using TestSetExtensions
julia> @includetests MyModule Set1
```

Though it would need to set `Base.ARGS` and then include `runtests.jl`so that any setup gets done, and also would only work if the tests were written using the `@includetests` macro.

-s


On Mon, Oct 31, 2016, at 08:54 PM, mmus wrote:
My biggest problem with Base.Test is that you can't individually select which  testsets you want to run
i.e. Pkg.test("MyModule","Set1")
which makes testing really annoying.

This package addresses this issue (albeit a little differently) 

I do believe having a more featured Base.Test with a few more batteries included is an overall boon.


On Monday, October 31, 2016 at 12:25:47 PM UTC-4, Spencer Russell wrote:
I'm not planning on merging this functionality into Base.Test. I think it's nice to keep Base.Test pretty minimal but extensible, and then have extra functionality provided via packages so they're easier for the community to iterate on and contribute to.

-s


On Mon, Oct 31, 2016, at 02:17 AM, mmus wrote:
Will these improvements make into Base.Test ?

On Monday, October 31, 2016 at 12:53:37 AM UTC-4, Spencer Russell wrote:
I think in general the culture in the Julia community is very pro-testing, which I really appreciate. I saw your post recently about PyTest, but I’m generally pretty happy with the built-in `@testset` / `@test` system, and just wanted some lightweight convenience functionality that wouldn’t require people to restructure their existing tests.

-s


On Oct 29, 2016, at 4:30 AM, [hidden email] wrote:

Hey Spencer,

I like this! 

I have recently started a testing-related package as well, maybe you've noticed (https://github.com/pdobacz/PyTest.jl). It seems that both packages could work together and complement quite seamlessly, so I'll definitely give yours a try. 

BTW, do you think there is much demand for Julia testing tools?