I was promised a debugger

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

I was promised a debugger

Andreas Lobinger
Hello colleagues,

the end of the year is approaching and for some of us this means planning the work for next year. I'm following now the julia development for more than 2 years and for me it would make some sense to introduce julia to my professional work also. Which is dominated by matlab use right now. So we have expectations what a programming environment should provide.

Unfortunately a debugger is contained in that expectation.  And i miss to see progress.

But that's the question here: Is there some kind of hidden work on a debugger, that doesn't show up here or over there at julia-users?

Wishing a happy day,
        Andreas



Reply | Threaded
Open this post in threaded view
|

Re: I was promised a debugger

Tim Holy
https://github.com/Keno/Gallium.jl
https://www.youtube.com/watch?v=Kv6TO_gm2yw

But building is not for the faint of heart. The issue that matters is
https://github.com/JuliaLang/julia/issues/9336, and you can subscribe to it if
you want to follow the progress. This should also begin to give you a sense of
what a massive undertaking this is, as it has required quite a few changes to
LLVM.

(It would be great if Keno's patches got reviewed over at LLVM more quickly.)

Best,
--Tim

On Wednesday, December 02, 2015 01:53:51 AM Andreas Lobinger wrote:

> Hello colleagues,
>
> the end of the year is approaching and for some of us this means planning
> the work for next year. I'm following now the julia development for more
> than 2 years and for me it would make some sense to introduce julia to my
> professional work also. Which is dominated by matlab use right now. So we
> have expectations what a programming environment should provide.
>
> Unfortunately a debugger is contained in that expectation.  And i miss to
> see progress.
>
> But that's the question here: Is there some kind of hidden work on a
> debugger, that doesn't show up here or over there at julia-users?
>
> Wishing a happy day,
>         Andreas

Reply | Threaded
Open this post in threaded view
|

Re: I was promised a debugger

Andreas Lobinger
I clearly understand that this is complicated stuff and will take some time until finished. Still it's frustrating not being able to use the prototype of Gallium.jl as the build instructions do not work out-of-the-box. What would you recommend to do to get a 'working' build?
Reply | Threaded
Open this post in threaded view
|

Re: I was promised a debugger

Jacob Quinn
I think Jon Malmaud had built a Docker image that included everything needed to have a working Gallium build. You might search the archives here and see what you find.

-Jacob

On Wed, Dec 2, 2015 at 5:47 AM, Andreas Lobinger <[hidden email]> wrote:
I clearly understand that this is complicated stuff and will take some time until finished. Still it's frustrating not being able to use the prototype of Gallium.jl as the build instructions do not work out-of-the-box. What would you recommend to do to get a 'working' build?

Reply | Threaded
Open this post in threaded view
|

Re: I was promised a debugger

Jeffrey Sarnoff
that link

On Wednesday, December 2, 2015 at 12:37:52 PM UTC-5, Jacob Quinn wrote:
I think Jon Malmaud had built a Docker image that included everything needed to have a working Gallium build. You might search the archives here and see what you find.

-Jacob

On Wed, Dec 2, 2015 at 5:47 AM, Andreas Lobinger <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="2JIpgE1OCgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">lobi...@...> wrote:
I clearly understand that this is complicated stuff and will take some time until finished. Still it's frustrating not being able to use the prototype of Gallium.jl as the build instructions do not work out-of-the-box. What would you recommend to do to get a 'working' build?

Reply | Threaded
Open this post in threaded view
|

Re: I was promised a debugger

Viral Shah
In reply to this post by Tim Holy
Keno is working hard on the debugger, and this is definitely the number one request from a lot of people. A lot of progress is planned over Dec and Jan.

-viral

On Wednesday, December 2, 2015 at 5:40:39 PM UTC+5:30, Tim Holy wrote:
<a href="https://github.com/Keno/Gallium.jl" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2FKeno%2FGallium.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFygrOY9ag_wNJAbSzfUjA3gy71TQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2FKeno%2FGallium.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFygrOY9ag_wNJAbSzfUjA3gy71TQ&#39;;return true;">https://github.com/Keno/Gallium.jl
<a href="https://www.youtube.com/watch?v=Kv6TO_gm2yw" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.youtube.com/watch?v\75Kv6TO_gm2yw&#39;;return true;" onclick="this.href=&#39;https://www.youtube.com/watch?v\75Kv6TO_gm2yw&#39;;return true;">https://www.youtube.com/watch?v=Kv6TO_gm2yw

But building is not for the faint of heart. The issue that matters is
<a href="https://github.com/JuliaLang/julia/issues/9336" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2FJuliaLang%2Fjulia%2Fissues%2F9336\46sa\75D\46sntz\0751\46usg\75AFQjCNG8y8groi003X78GUNEMdfGgQcQhA&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2FJuliaLang%2Fjulia%2Fissues%2F9336\46sa\75D\46sntz\0751\46usg\75AFQjCNG8y8groi003X78GUNEMdfGgQcQhA&#39;;return true;">https://github.com/JuliaLang/julia/issues/9336, and you can subscribe to it if
you want to follow the progress. This should also begin to give you a sense of
what a massive undertaking this is, as it has required quite a few changes to
LLVM.

(It would be great if Keno's patches got reviewed over at LLVM more quickly.)

Best,
--Tim

On Wednesday, December 02, 2015 01:53:51 AM Andreas Lobinger wrote:

> Hello colleagues,
>
> the end of the year is approaching and for some of us this means planning
> the work for next year. I'm following now the julia development for more
> than 2 years and for me it would make some sense to introduce julia to my
> professional work also. Which is dominated by matlab use right now. So we
> have expectations what a programming environment should provide.
>
> Unfortunately a debugger is contained in that expectation.  And i miss to
> see progress.
>
> But that's the question here: Is there some kind of hidden work on a
> debugger, that doesn't show up here or over there at julia-users?
>
> Wishing a happy day,
>         Andreas

Reply | Threaded
Open this post in threaded view
|

Re: I was promised a debugger

Michael Turok
Viral - can you please share the specifics of the plan?

On Saturday, December 5, 2015 at 4:20:57 AM UTC-5, Viral Shah wrote:
Keno is working hard on the debugger, and this is definitely the number one request from a lot of people. A lot of progress is planned over Dec and Jan.

-viral

On Wednesday, December 2, 2015 at 5:40:39 PM UTC+5:30, Tim Holy wrote:
<a href="https://github.com/Keno/Gallium.jl" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2FKeno%2FGallium.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFygrOY9ag_wNJAbSzfUjA3gy71TQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2FKeno%2FGallium.jl\46sa\75D\46sntz\0751\46usg\75AFQjCNFygrOY9ag_wNJAbSzfUjA3gy71TQ&#39;;return true;">https://github.com/Keno/Gallium.jl
<a href="https://www.youtube.com/watch?v=Kv6TO_gm2yw" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.youtube.com/watch?v\75Kv6TO_gm2yw&#39;;return true;" onclick="this.href=&#39;https://www.youtube.com/watch?v\75Kv6TO_gm2yw&#39;;return true;">https://www.youtube.com/watch?v=Kv6TO_gm2yw

But building is not for the faint of heart. The issue that matters is
<a href="https://github.com/JuliaLang/julia/issues/9336" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2FJuliaLang%2Fjulia%2Fissues%2F9336\46sa\75D\46sntz\0751\46usg\75AFQjCNG8y8groi003X78GUNEMdfGgQcQhA&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2FJuliaLang%2Fjulia%2Fissues%2F9336\46sa\75D\46sntz\0751\46usg\75AFQjCNG8y8groi003X78GUNEMdfGgQcQhA&#39;;return true;">https://github.com/JuliaLang/julia/issues/9336, and you can subscribe to it if
you want to follow the progress. This should also begin to give you a sense of
what a massive undertaking this is, as it has required quite a few changes to
LLVM.

(It would be great if Keno's patches got reviewed over at LLVM more quickly.)

Best,
--Tim

On Wednesday, December 02, 2015 01:53:51 AM Andreas Lobinger wrote:

> Hello colleagues,
>
> the end of the year is approaching and for some of us this means planning
> the work for next year. I'm following now the julia development for more
> than 2 years and for me it would make some sense to introduce julia to my
> professional work also. Which is dominated by matlab use right now. So we
> have expectations what a programming environment should provide.
>
> Unfortunately a debugger is contained in that expectation.  And i miss to
> see progress.
>
> But that's the question here: Is there some kind of hidden work on a
> debugger, that doesn't show up here or over there at julia-users?
>
> Wishing a happy day,
>         Andreas

Reply | Threaded
Open this post in threaded view
|

Re: I was promised a debugger

Keno Fischer
Hi everyone,

just wanted to give a little bit more context. The link earlier in this thread shows the Gallium preview
as it was in September. In terms of features to Gallium, I have not added any more functionality since.
My primary focus instead has been on making it actually possible/easy to install, because the whole
custom julia/llvm version thing was just a mess. Separately, we really need to switch julia to a newer
LLVM anyway due to bugfixes, ARM support, and a whole slew of other improvements. Doing this
transition was a lot of work, but all the code is now in place. The julia side of those changes is currently
under review at https://github.com/JuliaLang/julia/pull/14272. The LLVM side of those changes in mostly
merged upstream, save for http://reviews.llvm.org/D15202 which I am expecting to have merged soon.
The plan here is to switch julia master over to LLVM 3.7.1 (plus patches like the one I mentioned) sometime
in the next few days.

Once that is done (expected this week), I will be back to fully working on the debugger, but this time around,
everything should hopefully be quite easy to install and try out for everybody. There is a few separate areas
that need to be worked on. One is debug info emission/propagation in LLVM. We have worked with some
folks to start improving debug info quality (the result of which is at http://reviews.llvm.org/D11933 - was
approved and got merged, but caused some regression on other platforms that are currently being worked
out). As part of this, I am also hoping to add a systematic multi-location debug info framework to LLVM, in
order to be able to encode all the information we have in julia (for examples, arguments are generally available
both individually and by unboxing it again from the argument array, and it is not clear which has longer lifetime).

The other major piece of work is the user interaction and usage model. My eventual plan is to have the
debugger follow a hybrid model where it uses a native debugger for setting breakpoints/obtaining variable
information initially, but where the stepping happens in an AST based debugger to make sure that you retain
the environments full information while stepping.

One of the primary questions is what the best path forward towards this goal is that will also allow people to
start using it as quickly as possible. A big part of the complexity of Gallium is that it uses LLDB to do all the
heavy lifting, and while using LLDB is great for all sorts of other things (remote debugging, separate process,
 integration with C/C++ debugging, etc), it may be getting in the way of iterating quickly to get something usable. 
There are also concerns about the quality of LLDB's codebase (it feels rather bloated and significantly lower quality
than the rest of LLVM/Clang). As a result, I am actually planning to temporarily move away from the LLDB
dependence. I already have a toy DWARF reader written in pure julia at https://github.com/Keno/DWARF.jl and
Gallium actually already makes partial use of that, so the immediate plan is to disable the LLDB dependency in
Gallium, which will disable the nice features I mentioned above (and probably be lower performance in debugging,
- i.e. don't try to debug your millions of line julia projects), but hopefully should make iteration a lot faster
and allow us to make sure we emit sufficiently accurate and high quality debug information, allow us to experiment
with the user interface. Then, once we have something we're happy with, we can decide whether it makes sense
to re-enable the use of LLDB, or rewrite those features ourselves in julia (I don't know the answer to that question,
which is why I really like this approach which allows us to delay the decision).


Keno

On Fri, Dec 11, 2015 at 7:55 PM, Michael Turok <[hidden email]> wrote:
Viral - can you please share the specifics of the plan?


On Saturday, December 5, 2015 at 4:20:57 AM UTC-5, Viral Shah wrote:
Keno is working hard on the debugger, and this is definitely the number one request from a lot of people. A lot of progress is planned over Dec and Jan.

-viral

On Wednesday, December 2, 2015 at 5:40:39 PM UTC+5:30, Tim Holy wrote:
https://github.com/Keno/Gallium.jl
https://www.youtube.com/watch?v=Kv6TO_gm2yw

But building is not for the faint of heart. The issue that matters is
https://github.com/JuliaLang/julia/issues/9336, and you can subscribe to it if
you want to follow the progress. This should also begin to give you a sense of
what a massive undertaking this is, as it has required quite a few changes to
LLVM.

(It would be great if Keno's patches got reviewed over at LLVM more quickly.)

Best,
--Tim

On Wednesday, December 02, 2015 01:53:51 AM Andreas Lobinger wrote:

> Hello colleagues,
>
> the end of the year is approaching and for some of us this means planning
> the work for next year. I'm following now the julia development for more
> than 2 years and for me it would make some sense to introduce julia to my
> professional work also. Which is dominated by matlab use right now. So we
> have expectations what a programming environment should provide.
>
> Unfortunately a debugger is contained in that expectation.  And i miss to
> see progress.
>
> But that's the question here: Is there some kind of hidden work on a
> debugger, that doesn't show up here or over there at julia-users?
>
> Wishing a happy day,
>         Andreas


Reply | Threaded
Open this post in threaded view
|

Re: I was promised a debugger

Andreas Lobinger
In reply to this post by Viral Shah
Hello colleague,

On Saturday, December 5, 2015 at 10:20:57 AM UTC+1, Viral Shah wrote:
Keno is working hard on the debugger, and this is definitely the number one request from a lot of people. A lot of progress is planned over Dec and Jan.

If my calendar is working correctly, we have reached March. Is there an update on that?

Wishing a happy day,
         Andreas
Reply | Threaded
Open this post in threaded view
|

Re: I was promised a debugger

Viral Shah
On Wednesday, March 9, 2016 at 6:09:05 PM UTC+5:30, Andreas Lobinger wrote:
Hello colleague,

On Saturday, December 5, 2015 at 10:20:57 AM UTC+1, Viral Shah wrote:
Keno is working hard on the debugger, and this is definitely the number one request from a lot of people. A lot of progress is planned over Dec and Jan.

If my calendar is working correctly, we have reached March. Is there an update on that?

Yes, your calendar is working correctly :-) We are roughly on track to have a first cut of a usable debugger in March. We'll say something here as soon as it is ready for more people to try out. We are also aiming for 0.5 to ship with the debugger.

-viral

Reply | Threaded
Open this post in threaded view
|

RE: I was promised a debugger

David Anthoff

So here is what I’m hearing:

 

Stefan yesterday wrote an email that he thinks the array changes (without the view-for-slices stuff) should be wrapped up and then a RC for 0.5 be declared. To me that suggested that some of the issues on the 0.5 milestone on github would be moved to a later release.

 

Jeff yesterday wrote an email saying that the issue tracker reflects the best understanding of the core of what will go into 0.5. Note that there is e.g. no issue for a debugger in that list, but a fair number of things that to me look like stuff above and beyond finishing the array stuff.

 

Viral today writes that the debugger will be part of 0.5, which to me suggest quite a bit more work than Stefan’s suggestion or what is tracked on github for 0.5.

 

I don’t mind that there is no clear agreed plan for what 0.5 should be, 0.4 works just fine for my needs and whenever 0.5 drops, I’ll be happy. But I do think that a more consistent communication of plans would be helpful for many users. And if there is no clear, agreed plan, then just writing that in response to queries seems like a better idea than sending out three different timelines :)

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Viral Shah
Sent: Wednesday, March 9, 2016 9:48 PM
To: julia-dev <[hidden email]>
Subject: Re: [julia-dev] I was promised a debugger

 

On Wednesday, March 9, 2016 at 6:09:05 PM UTC+5:30, Andreas Lobinger wrote:

Hello colleague,

On Saturday, December 5, 2015 at 10:20:57 AM UTC+1, Viral Shah wrote:

Keno is working hard on the debugger, and this is definitely the number one request from a lot of people. A lot of progress is planned over Dec and Jan.


If my calendar is working correctly, we have reached March. Is there an update on that?

 

Yes, your calendar is working correctly :-) We are roughly on track to have a first cut of a usable debugger in March. We'll say something here as soon as it is ready for more people to try out. We are also aiming for 0.5 to ship with the debugger.

 

-viral

 

Reply | Threaded
Open this post in threaded view
|

Re: I was promised a debugger

Viral Shah
That reading of emails is a bit too legal! I am sure you already know that releases are a collective decision, that involve many more contributors than the three of us.

Some issues will be moved to beyond 0.5, the array changes (except for the big one) are mostly done, and the debugger is ongoing work (with many PRs already merged). Rather than reading this as an inconsistency which it is not, I suggest that we consider this thread as a start of discussion on what 0.5 could potentially look like. I personally see much more clarity in the 0.5 plan than there was 3 months ago, which clearly is progress.

-viral

> On 10-Mar-2016, at 11:41 PM, Viral Shah <[hidden email]> wrote:
>
> Ok - that reading of emails is a bit too legal! I am sure you already know that releases are a collective decision, that involve many more contributors than the three of us.
>
> Some issues will be moved to beyond 0.5, the array changes (except for the big one) are mostly done, and the debugger is ongoing work (with many PRs already merged). Rather than reading this as inconsistency, I suggest that we consider this thread as a start of discussion on what 0.5 could potentially look like. I personally see much more clarity in the 0.5 plan than there was 3 months ago, which clearly is progress.
>
> -viral
>
>
>
>> On 10-Mar-2016, at 11:29 PM, David Anthoff <[hidden email]> wrote:
>>
>> So here is what I’m hearing:
>>
>> Stefan yesterday wrote an email that he thinks the array changes (without the view-for-slices stuff) should be wrapped up and then a RC for 0.5 be declared. To me that suggested that some of the issues on the 0.5 milestone on github would be moved to a later release.
>>
>> Jeff yesterday wrote an email saying that the issue tracker reflects the best understanding of the core of what will go into 0.5. Note that there is e.g. no issue for a debugger in that list, but a fair number of things that to me look like stuff above and beyond finishing the array stuff.
>>
>> Viral today writes that the debugger will be part of 0.5, which to me suggest quite a bit more work than Stefan’s suggestion or what is tracked on github for 0.5.
>>
>> I don’t mind that there is no clear agreed plan for what 0.5 should be, 0.4 works just fine for my needs and whenever 0.5 drops, I’ll be happy. But I do think that a more consistent communication of plans would be helpful for many users. And if there is no clear, agreed plan, then just writing that in response to queries seems like a better idea than sending out three different timelines :)
>>
>> From: [hidden email] [mailto:[hidden email]] On Behalf Of Viral Shah
>> Sent: Wednesday, March 9, 2016 9:48 PM
>> To: julia-dev <[hidden email]>
>> Subject: Re: [julia-dev] I was promised a debugger
>>
>> On Wednesday, March 9, 2016 at 6:09:05 PM UTC+5:30, Andreas Lobinger wrote:
>>> Hello colleague,
>>>
>>> On Saturday, December 5, 2015 at 10:20:57 AM UTC+1, Viral Shah wrote:
>>>> Keno is working hard on the debugger, and this is definitely the number one request from a lot of people. A lot of progress is planned over Dec and Jan.
>>>
>>> If my calendar is working correctly, we have reached March. Is there an update on that?
>>
>> Yes, your calendar is working correctly :-) We are roughly on track to have a first cut of a usable debugger in March. We'll say something here as soon as it is ready for more people to try out. We are also aiming for 0.5 to ship with the debugger.
>>
>> -viral
>




-viral