Quantcast

Immutable type modification of SparseMatrixCSC

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Immutable type modification of SparseMatrixCSC

Corbin Foucart
The SparseMatrixCSC type is immutable, which I understand to mean that I must respect the types of the attributes of SparseMatrixCSC types, as well as not changing the attributes themselves.

The following gives a loadError (type is immutable). I am confused that I can modify the colptr and rowval attributes of A, but not the A.n attribute. Why is this? I am trying to change the size of a sparse matrix after removing the data from a row and column.

I have checked that both A.n and A.n - 1 are type Int.

function rmCol(A::SparseMatrixCSC, rmCol::Int)
    colRmRange = A.colptr[rmCol]:(A.colptr[rmCol+1]-1)
    filter!(e -> e != rmCol, A.colptr)
    
    deleteat!(A.rowval, rowval[colRmRange])
    deleteat!(A.nzval, colRmRange)
    A.n -= 1 # inclusion of this line throws error
    
end
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Immutable type modification of SparseMatrixCSC

Jared Crean
When a type is immutable, that means you can't modify the data in the type.  If a field of a type is mutable, a reference to it is stored in the type, and you cannot modify that reference.  In the case of SparseMatrixCSC, that means you can't make A.colptr point to a different array, for example.  You are still free to change the values in A.colptr, because that does not require modifying the reference.  If the field is immutable, then the value itself is stored in the type, not a reference to the value, and you cannot modify the value.  That's why you get the error in the example.

The best solution I can think of for your rmCol example is to create a new SparseMatrixCSC that uses the same arrays as the old one (to avoid increasing memory usage) and return it.  Replacing the last line of rmCol with:

  SparseMatrixCSC(m, n-1, A.colptr, A.rowval, A.nzval)

  should do the trick.  The only sticky point is that the original A now contains invalid data, so you'll have to make sure only the the matrix returned from rmCol is used in the future, not the original A.

  Jared Crean


On Saturday, November 5, 2016 at 5:07:45 PM UTC-4, Corbin Foucart wrote:
The SparseMatrixCSC type is immutable, which I understand to mean that I must respect the types of the attributes of SparseMatrixCSC types, as well as not changing the attributes themselves.

The following gives a loadError (type is immutable). I am confused that I can modify the colptr and rowval attributes of A, but not the A.n attribute. Why is this? I am trying to change the size of a sparse matrix after removing the data from a row and column.

I have checked that both A.n and A.n - 1 are type Int.

function rmCol(A::SparseMatrixCSC, rmCol::Int)
    colRmRange = A.colptr[rmCol]:(A.colptr[rmCol+1]-1)
    filter!(e -> e != rmCol, A.colptr)
    
    deleteat!(A.rowval, rowval[colRmRange])
    deleteat!(A.nzval, colRmRange)
    A.n -= 1 # inclusion of this line throws error
    
end
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Immutable type modification of SparseMatrixCSC

Corbin Foucart
I see. returning a new, smaller instance of the matrix was my temporary solution. Bummer that it's necessary... Is it really so strange to want to change the size of a 

Another alternative which occurs to me is to make some child class of SparseCSCMatrix with two more arrays, each with one element, which track the sizes? Since I am making the matrix smaller, not larger, I can't imagine that this would be an issue. However, if I need to allocate two large sparse matrices just to remove a row/col pair, then I can suddenly use half the memory I could before...


On Saturday, November 5, 2016 at 7:42:55 PM UTC-4, Jared Crean wrote:
When a type is immutable, that means you can't modify the data in the type.  If a field of a type is mutable, a reference to it is stored in the type, and you cannot modify that reference.  In the case of SparseMatrixCSC, that means you can't make A.colptr point to a different array, for example.  You are still free to change the values in A.colptr, because that does not require modifying the reference.  If the field is immutable, then the value itself is stored in the type, not a reference to the value, and you cannot modify the value.  That's why you get the error in the example.

The best solution I can think of for your rmCol example is to create a new SparseMatrixCSC that uses the same arrays as the old one (to avoid increasing memory usage) and return it.  Replacing the last line of rmCol with:

  SparseMatrixCSC(m, n-1, A.colptr, A.rowval, A.nzval)

  should do the trick.  The only sticky point is that the original A now contains invalid data, so you'll have to make sure only the the matrix returned from rmCol is used in the future, not the original A.

  Jared Crean


On Saturday, November 5, 2016 at 5:07:45 PM UTC-4, Corbin Foucart wrote:
The SparseMatrixCSC type is immutable, which I understand to mean that I must respect the types of the attributes of SparseMatrixCSC types, as well as not changing the attributes themselves.

The following gives a loadError (type is immutable). I am confused that I can modify the colptr and rowval attributes of A, but not the A.n attribute. Why is this? I am trying to change the size of a sparse matrix after removing the data from a row and column.

I have checked that both A.n and A.n - 1 are type Int.

function rmCol(A::SparseMatrixCSC, rmCol::Int)
    colRmRange = A.colptr[rmCol]:(A.colptr[rmCol+1]-1)
    filter!(e -> e != rmCol, A.colptr)
    
    deleteat!(A.rowval, rowval[colRmRange])
    deleteat!(A.nzval, colRmRange)
    A.n -= 1 # inclusion of this line throws error
    
end
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Immutable type modification of SparseMatrixCSC

vavasis
Just to clarify, you don't actually need to allocate the memory for two large sparse matrices.  You can change colptr, rowval, and nzval in place (as you have already done) and then create a new sparse matrix using these modified arrays.  In this case, the arrays won't be copied over; instead the new sparse matrix will get a pointer to the already-allocated arrays.  This technique could lead to some subtle program bugs (because if two sparse matrices share an nzval array, then any change to one affects the other), but you can guard against this by making the old matrix's name inaccessible, for instance, by giving the new matrix the same name as the old one.

-- Steve Vavasis


On Saturday, November 5, 2016 at 8:32:58 PM UTC-4, Corbin Foucart wrote:
I see. returning a new, smaller instance of the matrix was my temporary solution. Bummer that it's necessary... Is it really so strange to want to change the size of a 

Another alternative which occurs to me is to make some child class of SparseCSCMatrix with two more arrays, each with one element, which track the sizes? Since I am making the matrix smaller, not larger, I can't imagine that this would be an issue. However, if I need to allocate two large sparse matrices just to remove a row/col pair, then I can suddenly use half the memory I could before...


On Saturday, November 5, 2016 at 7:42:55 PM UTC-4, Jared Crean wrote:
When a type is immutable, that means you can't modify the data in the type.  If a field of a type is mutable, a reference to it is stored in the type, and you cannot modify that reference.  In the case of SparseMatrixCSC, that means you can't make A.colptr point to a different array, for example.  You are still free to change the values in A.colptr, because that does not require modifying the reference.  If the field is immutable, then the value itself is stored in the type, not a reference to the value, and you cannot modify the value.  That's why you get the error in the example.

The best solution I can think of for your rmCol example is to create a new SparseMatrixCSC that uses the same arrays as the old one (to avoid increasing memory usage) and return it.  Replacing the last line of rmCol with:

  SparseMatrixCSC(m, n-1, A.colptr, A.rowval, A.nzval)

  should do the trick.  The only sticky point is that the original A now contains invalid data, so you'll have to make sure only the the matrix returned from rmCol is used in the future, not the original A.

  Jared Crean


On Saturday, November 5, 2016 at 5:07:45 PM UTC-4, Corbin Foucart wrote:
The SparseMatrixCSC type is immutable, which I understand to mean that I must respect the types of the attributes of SparseMatrixCSC types, as well as not changing the attributes themselves.

The following gives a loadError (type is immutable). I am confused that I can modify the colptr and rowval attributes of A, but not the A.n attribute. Why is this? I am trying to change the size of a sparse matrix after removing the data from a row and column.

I have checked that both A.n and A.n - 1 are type Int.

function rmCol(A::SparseMatrixCSC, rmCol::Int)
    colRmRange = A.colptr[rmCol]:(A.colptr[rmCol+1]-1)
    filter!(e -> e != rmCol, A.colptr)
    
    deleteat!(A.rowval, rowval[colRmRange])
    deleteat!(A.nzval, colRmRange)
    A.n -= 1 # inclusion of this line throws error
    
end
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Immutable type modification of SparseMatrixCSC

Milan Bouchet-Valat
For reference, here's the long discussion that happened before making
that change:
https://github.com/JuliaLang/julia/pull/16371

Indeed I think Tony was right that this has undesirable consequences in
terms of usability. Not being able to use the same in-place API for
dense and sparse matrices is a no-go IMHO.


Regards

Le samedi 05 novembre 2016 à 20:11 -0700, [hidden email] a
écrit :

> Just to clarify, you don't actually need to allocate the memory for
> two large sparse matrices.  You can change colptr, rowval, and nzval
> in place (as you have already done) and then create a new sparse
> matrix using these modified arrays.  In this case, the arrays won't
> be copied over; instead the new sparse matrix will get a pointer to
> the already-allocated arrays.  This technique could lead to some
> subtle program bugs (because if two sparse matrices share an nzval
> array, then any change to one affects the other), but you can guard
> against this by making the old matrix's name inaccessible, for
> instance, by giving the new matrix the same name as the old one.
>
> -- Steve Vavasis
>
>
> > I see. returning a new, smaller instance of the matrix was my
> > temporary solution. Bummer that it's necessary... Is it really so
> > strange to want to change the size of a 
> >
> > Another alternative which occurs to me is to make some child class
> > of SparseCSCMatrix with two more arrays, each with one element,
> > which track the sizes? Since I am making the matrix smaller, not
> > larger, I can't imagine that this would be an issue. However, if I
> > need to allocate two large sparse matrices just to remove a row/col
> > pair, then I can suddenly use half the memory I could before...
> >
> >
> > > When a type is immutable, that means you can't modify the data in
> > > the type.  If a field of a type is mutable, a reference to it is
> > > stored in the type, and you cannot modify that reference.  In the
> > > case of SparseMatrixCSC, that means you can't make A.colptr point
> > > to a different array, for example.  You are still free to change
> > > the values in A.colptr, because that does not require modifying
> > > the reference.  If the field is immutable, then the value itself
> > > is stored in the type, not a reference to the value, and you
> > > cannot modify the value.  That's why you get the error in the
> > > example.
> > >
> > > The best solution I can think of for your rmCol example is to
> > > create a new SparseMatrixCSC that uses the same arrays as the old
> > > one (to avoid increasing memory usage) and return it.  Replacing
> > > the last line of rmCol with:
> > >
> > >   SparseMatrixCSC(m, n-1, A.colptr, A.rowval, A.nzval)
> > >
> > >   should do the trick.  The only sticky point is that the
> > > original A now contains invalid data, so you'll have to make sure
> > > only the the matrix returned from rmCol is used in the future,
> > > not the original A.
> > >
> > >   Jared Crean
> > >
> > >
> > > > The SparseMatrixCSC type is immutable, which I understand to
> > > > mean that I must respect the types of the attributes of
> > > > SparseMatrixCSC types, as well as not changing the attributes
> > > > themselves.
> > > >
> > > > The following gives a loadError (type is immutable).  I am
> > > > confused that I can modify the colptr and rowval attributes of
> > > > A, but not the A.n attribute. Why is this? I am trying to
> > > > change the size of a sparse matrix after removing the data from
> > > > a row and column.
> > > >
> > > > I have checked that both A.n and A.n - 1 are type Int.
> > > >
> > > > function rmCol(A::SparseMatrixCSC, rmCol::Int)
> > > >     colRmRange = A.colptr[rmCol]:(A.colptr[rmCol+1]-1)
> > > >     filter!(e -> e != rmCol, A.colptr)
> > > >     
> > > >     deleteat!(A.rowval, rowval[colRmRange])
> > > >     deleteat!(A.nzval, colRmRange)
> > > >     A.n -= 1 # inclusion of this line throws error
> > > >     
> > > > end
> > > >
> > >
> >
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Immutable type modification of SparseMatrixCSC

Kristoffer Carlsson


On Sunday, November 6, 2016 at 2:25:27 AM UTC-8, Milan Bouchet-Valat wrote:
For reference, here's the long discussion that happened before making
that change:
<a href="https://github.com/JuliaLang/julia/pull/16371" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2FJuliaLang%2Fjulia%2Fpull%2F16371\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEROFjDJiXCSgaMNIalMyEH0ZGfmA&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2FJuliaLang%2Fjulia%2Fpull%2F16371\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEROFjDJiXCSgaMNIalMyEH0ZGfmA&#39;;return true;">https://github.com/JuliaLang/julia/pull/16371

Indeed I think Tony was right that this has undesirable consequences in
terms of usability. Not being able to use the same in-place API for
dense and sparse matrices is a no-go IMHO.


Please elaborate. You cannot resize dense matrices in-place either.

Just create a new matrix from the fields of the old one if you want to change the size. How does this limit usability?

 

Regards

Le samedi 05 novembre 2016 à 20:11 -0700, <a href="javascript:" target="_blank" gdf-obfuscated-mailto="2PJnkbYSBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">vav...@... a
écrit :

> Just to clarify, you don't actually need to allocate the memory for
> two large sparse matrices.  You can change colptr, rowval, and nzval
> in place (as you have already done) and then create a new sparse
> matrix using these modified arrays.  In this case, the arrays won't
> be copied over; instead the new sparse matrix will get a pointer to
> the already-allocated arrays.  This technique could lead to some
> subtle program bugs (because if two sparse matrices share an nzval
> array, then any change to one affects the other), but you can guard
> against this by making the old matrix's name inaccessible, for
> instance, by giving the new matrix the same name as the old one.
>
> -- Steve Vavasis
>
>
> > I see. returning a new, smaller instance of the matrix was my
> > temporary solution. Bummer that it's necessary... Is it really so
> > strange to want to change the size of a 
> >
> > Another alternative which occurs to me is to make some child class
> > of SparseCSCMatrix with two more arrays, each with one element,
> > which track the sizes? Since I am making the matrix smaller, not
> > larger, I can't imagine that this would be an issue. However, if I
> > need to allocate two large sparse matrices just to remove a row/col
> > pair, then I can suddenly use half the memory I could before...
> >
> >
> > > When a type is immutable, that means you can't modify the data in
> > > the type.  If a field of a type is mutable, a reference to it is
> > > stored in the type, and you cannot modify that reference.  In the
> > > case of SparseMatrixCSC, that means you can't make A.colptr point
> > > to a different array, for example.  You are still free to change
> > > the values in A.colptr, because that does not require modifying
> > > the reference.  If the field is immutable, then the value itself
> > > is stored in the type, not a reference to the value, and you
> > > cannot modify the value.  That's why you get the error in the
> > > example.
> > >
> > > The best solution I can think of for your rmCol example is to
> > > create a new SparseMatrixCSC that uses the same arrays as the old
> > > one (to avoid increasing memory usage) and return it.  Replacing
> > > the last line of rmCol with:
> > >
> > >   SparseMatrixCSC(m, n-1, A.colptr, A.rowval, A.nzval)
> > >
> > >   should do the trick.  The only sticky point is that the
> > > original A now contains invalid data, so you'll have to make sure
> > > only the the matrix returned from rmCol is used in the future,
> > > not the original A.
> > >
> > >   Jared Crean
> > >
> > >
> > > > The SparseMatrixCSC type is immutable, which I understand to
> > > > mean that I must respect the types of the attributes of
> > > > SparseMatrixCSC types, as well as not changing the attributes
> > > > themselves.
> > > >
> > > > The following gives a loadError (type is immutable).  I am
> > > > confused that I can modify the colptr and rowval attributes of
> > > > A, but not the A.n attribute. Why is this? I am trying to
> > > > change the size of a sparse matrix after removing the data from
> > > > a row and column.
> > > >
> > > > I have checked that both A.n and A.n - 1 are type Int.
> > > >
> > > > function rmCol(A::SparseMatrixCSC, rmCol::Int)
> > > >     colRmRange = A.colptr[rmCol]:(A.colptr[rmCol+1]-1)
> > > >     filter!(e -> e != rmCol, A.colptr)
> > > >     
> > > >     deleteat!(A.rowval, rowval[colRmRange])
> > > >     deleteat!(A.nzval, colRmRange)
> > > >     A.n -= 1 # inclusion of this line throws error
> > > >     
> > > > end
> > > >
> > >
> >
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Immutable type modification of SparseMatrixCSC

Milan Bouchet-Valat
Le dimanche 06 novembre 2016 à 15:31 -0800, Kristoffer Carlsson a écrit :

>
>
> > For reference, here's the long discussion that happened before making 
> > that change: 
> > https://github.com/JuliaLang/julia/pull/16371 
> >
> > Indeed I think Tony was right that this has undesirable consequences in 
> > terms of usability. Not being able to use the same in-place API for 
> > dense and sparse matrices is a no-go IMHO. 
> >
>
> Please elaborate. You cannot resize dense matrices in-place either.
>
> Just create a new matrix from the fields of the old one if you want
> to change the size. How does this limit usability?
Right, I was being sloppy. That remark applies to sparse vectors, though.


Regards

 

> >
> > Regards 
> >
> > > > Le samedi 05 novembre 2016 à 20:11 -0700, [hidden email]
> > écrit : 
> > > Just to clarify, you don't actually need to allocate the memory for 
> > > two large sparse matrices.  You can change colptr, rowval, and nzval 
> > > in place (as you have already done) and then create a new sparse 
> > > matrix using these modified arrays.  In this case, the arrays won't 
> > > be copied over; instead the new sparse matrix will get a pointer to 
> > > the already-allocated arrays.  This technique could lead to some 
> > > subtle program bugs (because if two sparse matrices share an nzval 
> > > array, then any change to one affects the other), but you can guard 
> > > against this by making the old matrix's name inaccessible, for 
> > > instance, by giving the new matrix the same name as the old one. 
> > > 
> > > -- Steve Vavasis 
> > > 
> > > 
> > > > I see. returning a new, smaller instance of the matrix was my 
> > > > temporary solution. Bummer that it's necessary... Is it really so 
> > > > strange to want to change the size of a  
> > > > 
> > > > Another alternative which occurs to me is to make some child class 
> > > > of SparseCSCMatrix with two more arrays, each with one element, 
> > > > which track the sizes? Since I am making the matrix smaller, not 
> > > > larger, I can't imagine that this would be an issue. However, if I 
> > > > need to allocate two large sparse matrices just to remove a row/col 
> > > > pair, then I can suddenly use half the memory I could before... 
> > > > 
> > > > 
> > > > > When a type is immutable, that means you can't modify the data in 
> > > > > the type.  If a field of a type is mutable, a reference to it is 
> > > > > stored in the type, and you cannot modify that reference.  In the 
> > > > > case of SparseMatrixCSC, that means you can't make A.colptr point 
> > > > > to a different array, for example.  You are still free to change 
> > > > > the values in A.colptr, because that does not require modifying 
> > > > > the reference.  If the field is immutable, then the value itself 
> > > > > is stored in the type, not a reference to the value, and you 
> > > > > cannot modify the value.  That's why you get the error in the 
> > > > > example. 
> > > > > 
> > > > > The best solution I can think of for your rmCol example is to 
> > > > > create a new SparseMatrixCSC that uses the same arrays as the old 
> > > > > one (to avoid increasing memory usage) and return it.  Replacing 
> > > > > the last line of rmCol with: 
> > > > > 
> > > > >   SparseMatrixCSC(m, n-1, A.colptr, A.rowval, A.nzval) 
> > > > > 
> > > > >   should do the trick.  The only sticky point is that the 
> > > > > original A now contains invalid data, so you'll have to make sure 
> > > > > only the the matrix returned from rmCol is used in the future, 
> > > > > not the original A. 
> > > > > 
> > > > >   Jared Crean 
> > > > > 
> > > > > 
> > > > > > The SparseMatrixCSC type is immutable, which I understand to 
> > > > > > mean that I must respect the types of the attributes of 
> > > > > > SparseMatrixCSC types, as well as not changing the attributes 
> > > > > > themselves. 
> > > > > > 
> > > > > > The following gives a loadError (type is immutable).  I am 
> > > > > > confused that I can modify the colptr and rowval attributes of 
> > > > > > A, but not the A.n attribute. Why is this? I am trying to 
> > > > > > change the size of a sparse matrix after removing the data from 
> > > > > > a row and column. 
> > > > > > 
> > > > > > I have checked that both A.n and A.n - 1 are type Int. 
> > > > > > 
> > > > > > function rmCol(A::SparseMatrixCSC, rmCol::Int) 
> > > > > >     colRmRange = A.colptr[rmCol]:(A.colptr[rmCol+1]-1) 
> > > > > >     filter!(e -> e != rmCol, A.colptr) 
> > > > > >      
> > > > > >     deleteat!(A.rowval, rowval[colRmRange]) 
> > > > > >     deleteat!(A.nzval, colRmRange) 
> > > > > >     A.n -= 1 # inclusion of this line throws error 
> > > > > >      
> > > > > > end 
> > > > > > 
> > > > > 
> > > > 
Loading...