# Indexing problem Classic List Threaded 5 messages Reply | Threaded
Open this post in threaded view
|

## Indexing problem

 Hi,I'm an R user trying to learn Julia. I got hold of some code from the Knet package that I was playing around with. My goal is to set values to zero in a loop based on a logical expression, but I cannot figure out how the indexing works. Any help would be appreciated (the problem lies in w[1,(w.-(z))] = 0):using Knetpredict(w,x) = w*x .+ wlambda = 2z = Array{Float64}(1,13)loss(w,x,y) = sumabs2(y - predict(w,x)) / size(y,2)lossgradient = grad(loss)function train(w, data; lr=.1)    for (x,y) in data        dw = lossgradient(w, x, y)        z[:] = lr * lambda        w -= lr * dw        w -= lr * dw        w[1,(w.-(z))] = 0    end    return wendurl = "https://archive.ics.uci.edu/ml/machine-learning-databases/housing/housing.data"rawdata = readdlm(download(url))x = rawdata[:,1:13]'x = (x .- mean(x,2)) ./ std(x,2)y = rawdata[:,14:14]'w = Any[ 0.1*randn(1,13), 0 ]niter = 25lossest = zeros(niter)for i=1:niter; train(w, [(x,y)]); lossest[i]=loss(w,x,y); endBest regards,Patrik
Reply | Threaded
Open this post in threaded view
|

## Re: Indexing problem

 good things to know about how indexing worksThe indices for a Vector, or a column or row of a Matrix start at 1```length(avector)   # gets the number of elements in avectoravector        # gets the first item in avectoravector[end]      # gets the final item in avector avector[1:end]    # gets all elements of avectorint_column_vector = [10, 20, 30] 10 20 30int_column_vector 10# do not use zero as an indexint_column_vector[ 0 ]ERROR: BoundsError:# do not use false, true as indices because avec[ false ] means avec[ 0 ]```in ` w[1,(w.-(z))] = 0 `, the second index can simplify to `false`   (consider this)```avec = [ 10, 20, 30 ]avec1 = avec[ 1 ] avec1 == avec[ 1 + false ]avec2 = avec[  2 ]avec2 == avec[ 1 + true ] ```As a start, recheck indexing expressions, be more sure they do what you want them to do.On Wednesday, November 16, 2016 at 1:36:57 PM UTC-5, Patrik Waldmann wrote:Hi,I'm an R user trying to learn Julia. I got hold of some code from the Knet package that I was playing around with. My goal is to set values to zero in a loop based on a logical expression, but I cannot figure out how the indexing works. Any help would be appreciated (the problem lies in w[1,(w.-(z))] = 0):using Knetpredict(w,x) = w*x .+ wlambda = 2z = Array{Float64}(1,13)loss(w,x,y) = sumabs2(y - predict(w,x)) / size(y,2)lossgradient = grad(loss)function train(w, data; lr=.1)    for (x,y) in data        dw = lossgradient(w, x, y)        z[:] = lr * lambda        w -= lr * dw        w -= lr * dw        w[1,(w.-(z))] = 0    end    return wendurl = "https://archive.ics.uci.edu/ml/machine-learning-databases/housing/housing.data"rawdata = readdlm(download(url))x = rawdata[:,1:13]'x = (x .- mean(x,2)) ./ std(x,2)y = rawdata[:,14:14]'w = Any[ 0.1*randn(1,13), 0 ]niter = 25lossest = zeros(niter)for i=1:niter; train(w, [(x,y)]); lossest[i]=loss(w,x,y); endBest regards,Patrik
Reply | Threaded
Open this post in threaded view
|

## Re: Indexing problem

 I guess I should have explained my problem clearer. If I run the code without w[1,(w.-(z))] = 0, and do:dump(w)Array{Any}((2,))  1: Array{Float64}((1,13)) [-0.681392 0.595298 … 0.893845 -3.5044]  2: Float64 22.447679788630705and println(w)[-0.681392 0.595298 -0.394906 0.776983 -1.11178 3.11679 -0.0984956 -2.18501 0.928204 -0.484802 -1.86844 0.893845 -3.5044]println(w[1,1])[-0.681392 0.595298 -0.394906 0.776983 -1.11178 3.11679 -0.0984956 -2.18501 0.928204 -0.484802 -1.86844 0.893845 -3.5044]println(w[1,:])Any[[-0.681392 0.595298 -0.394906 0.776983 -1.11178 3.11679 -0.0984956 -2.18501 0.928204 -0.484802 -1.86844 0.893845 -3.5044]]This is very confusing for an R user like me. How do I access the column indexes of w and apply the logical expression w[1,(w.-(z))] = 0 ?PatrikOn Thursday, November 17, 2016 at 6:58:14 AM UTC+1, Jeffrey Sarnoff wrote:good things to know about how indexing worksThe indices for a Vector, or a column or row of a Matrix start at 1```length(avector)   # gets the number of elements in avectoravector        # gets the first item in avectoravector[end]      # gets the final item in avector avector[1:end]    # gets all elements of avectorint_column_vector = [10, 20, 30] 10 20 30int_column_vector 10# do not use zero as an indexint_column_vector[ 0 ]ERROR: BoundsError:# do not use false, true as indices because avec[ false ] means avec[ 0 ]```in ` w[1,(w.-(z))] = 0 `, the second index can simplify to `false`   (consider this)```avec = [ 10, 20, 30 ]avec1 = avec[ 1 ] avec1 == avec[ 1 + false ]avec2 = avec[  2 ]avec2 == avec[ 1 + true ] ```As a start, recheck indexing expressions, be more sure they do what you want them to do.On Wednesday, November 16, 2016 at 1:36:57 PM UTC-5, Patrik Waldmann wrote:Hi,I'm an R user trying to learn Julia. I got hold of some code from the Knet package that I was playing around with. My goal is to set values to zero in a loop based on a logical expression, but I cannot figure out how the indexing works. Any help would be appreciated (the problem lies in w[1,(w.-(z))] = 0):using Knetpredict(w,x) = w*x .+ wlambda = 2z = Array{Float64}(1,13)loss(w,x,y) = sumabs2(y - predict(w,x)) / size(y,2)lossgradient = grad(loss)function train(w, data; lr=.1)    for (x,y) in data        dw = lossgradient(w, x, y)        z[:] = lr * lambda        w -= lr * dw        w -= lr * dw        w[1,(w.-(z))] = 0    end    return wendurl = "https://archive.ics.uci.edu/ml/machine-learning-databases/housing/housing.data"rawdata = readdlm(download(url))x = rawdata[:,1:13]'x = (x .- mean(x,2)) ./ std(x,2)y = rawdata[:,14:14]'w = Any[ 0.1*randn(1,13), 0 ]niter = 25lossest = zeros(niter)for i=1:niter; train(w, [(x,y)]); lossest[i]=loss(w,x,y); endBest regards,Patrik
Reply | Threaded
Open this post in threaded view
|

## Re: Indexing problem

 I'm not familiar with the package in question, but this line:w = Any[ 0.1*randn(1,13), 0 ]may be what is causing the problem. It is creating a 2-element Vector, the first element of which is a 1x13 Matrix, and the second element is a scalar 0. The analogous object in R would be:W = list(matrix(0.1*rnorm(13),nrow=1), 0)In Julia, extraneous dimensions have an implicit index of 1 (this is a matlab-ism, and may disappear in future), so w, w[1,1], w[1,1,1] are all identical (and equivalent to W[] in R). w[1,:] is a bit of an odd case in that it returns a 1-element Vector containing a Matrix, but would be equivalent to W in R.I think what you may want is actuallyw[(w.-(z))]which can be written more clearly asw[-z .< w .< z]-SimonOn Thursday, 17 November 2016 09:39:14 UTC, Patrik Waldmann wrote:I guess I should have explained my problem clearer. If I run the code without w[1,(w.-(z))] = 0, and do:dump(w)Array{Any}((2,))  1: Array{Float64}((1,13)) [-0.681392 0.595298 … 0.893845 -3.5044]  2: Float64 22.447679788630705and println(w)[-0.681392 0.595298 -0.394906 0.776983 -1.11178 3.11679 -0.0984956 -2.18501 0.928204 -0.484802 -1.86844 0.893845 -3.5044]println(w[1,1])[-0.681392 0.595298 -0.394906 0.776983 -1.11178 3.11679 -0.0984956 -2.18501 0.928204 -0.484802 -1.86844 0.893845 -3.5044]println(w[1,:])Any[[-0.681392 0.595298 -0.394906 0.776983 -1.11178 3.11679 -0.0984956 -2.18501 0.928204 -0.484802 -1.86844 0.893845 -3.5044]]This is very confusing for an R user like me. How do I access the column indexes of w and apply the logical expression w[1,(w.-(z))] = 0 ?PatrikOn Thursday, November 17, 2016 at 6:58:14 AM UTC+1, Jeffrey Sarnoff wrote:good things to know about how indexing worksThe indices for a Vector, or a column or row of a Matrix start at 1```length(avector)   # gets the number of elements in avectoravector        # gets the first item in avectoravector[end]      # gets the final item in avector avector[1:end]    # gets all elements of avectorint_column_vector = [10, 20, 30] 10 20 30int_column_vector 10# do not use zero as an indexint_column_vector[ 0 ]ERROR: BoundsError:# do not use false, true as indices because avec[ false ] means avec[ 0 ]```in ` w[1,(w.-(z))] = 0 `, the second index can simplify to `false`   (consider this)```avec = [ 10, 20, 30 ]avec1 = avec[ 1 ] avec1 == avec[ 1 + false ]avec2 = avec[  2 ]avec2 == avec[ 1 + true ] ```As a start, recheck indexing expressions, be more sure they do what you want them to do.On Wednesday, November 16, 2016 at 1:36:57 PM UTC-5, Patrik Waldmann wrote:Hi,I'm an R user trying to learn Julia. I got hold of some code from the Knet package that I was playing around with. My goal is to set values to zero in a loop based on a logical expression, but I cannot figure out how the indexing works. Any help would be appreciated (the problem lies in w[1,(w.-(z))] = 0):using Knetpredict(w,x) = w*x .+ wlambda = 2z = Array{Float64}(1,13)loss(w,x,y) = sumabs2(y - predict(w,x)) / size(y,2)lossgradient = grad(loss)function train(w, data; lr=.1)    for (x,y) in data        dw = lossgradient(w, x, y)        z[:] = lr * lambda        w -= lr * dw        w -= lr * dw        w[1,(w.-(z))] = 0    end    return wendurl = "https://archive.ics.uci.edu/ml/machine-learning-databases/housing/housing.data"rawdata = readdlm(download(url))x = rawdata[:,1:13]'x = (x .- mean(x,2)) ./ std(x,2)y = rawdata[:,14:14]'w = Any[ 0.1*randn(1,13), 0 ]niter = 25lossest = zeros(niter)for i=1:niter; train(w, [(x,y)]); lossest[i]=loss(w,x,y); endBest regards,Patrik
Reply | Threaded
Open this post in threaded view
|

## Re: Indexing problem

 OK, that works fine. Thanks. I think it would be a good idea to drop the matlab-ism in future versions.PatrikOn Thursday, November 17, 2016 at 11:51:09 AM UTC+1, Simon Byrne wrote:I'm not familiar with the package in question, but this line:w = Any[ 0.1*randn(1,13), 0 ]may be what is causing the problem. It is creating a 2-element Vector, the first element of which is a 1x13 Matrix, and the second element is a scalar 0. The analogous object in R would be:W = list(matrix(0.1*rnorm(13),nrow=1), 0)In Julia, extraneous dimensions have an implicit index of 1 (this is a matlab-ism, and may disappear in future), so w, w[1,1], w[1,1,1] are all identical (and equivalent to W[] in R). w[1,:] is a bit of an odd case in that it returns a 1-element Vector containing a Matrix, but would be equivalent to W in R.I think what you may want is actuallyw[(w.-(z))]which can be written more clearly asw[-z .< w .< z]-SimonOn Thursday, 17 November 2016 09:39:14 UTC, Patrik Waldmann wrote:I guess I should have explained my problem clearer. If I run the code without w[1,(w.-(z))] = 0, and do:dump(w)Array{Any}((2,))  1: Array{Float64}((1,13)) [-0.681392 0.595298 … 0.893845 -3.5044]  2: Float64 22.447679788630705and println(w)[-0.681392 0.595298 -0.394906 0.776983 -1.11178 3.11679 -0.0984956 -2.18501 0.928204 -0.484802 -1.86844 0.893845 -3.5044]println(w[1,1])[-0.681392 0.595298 -0.394906 0.776983 -1.11178 3.11679 -0.0984956 -2.18501 0.928204 -0.484802 -1.86844 0.893845 -3.5044]println(w[1,:])Any[[-0.681392 0.595298 -0.394906 0.776983 -1.11178 3.11679 -0.0984956 -2.18501 0.928204 -0.484802 -1.86844 0.893845 -3.5044]]This is very confusing for an R user like me. How do I access the column indexes of w and apply the logical expression w[1,(w.-(z))] = 0 ?PatrikOn Thursday, November 17, 2016 at 6:58:14 AM UTC+1, Jeffrey Sarnoff wrote:good things to know about how indexing worksThe indices for a Vector, or a column or row of a Matrix start at 1```length(avector)   # gets the number of elements in avectoravector        # gets the first item in avectoravector[end]      # gets the final item in avector avector[1:end]    # gets all elements of avectorint_column_vector = [10, 20, 30] 10 20 30int_column_vector 10# do not use zero as an indexint_column_vector[ 0 ]ERROR: BoundsError:# do not use false, true as indices because avec[ false ] means avec[ 0 ]```in ` w[1,(w.-(z))] = 0 `, the second index can simplify to `false`   (consider this)```avec = [ 10, 20, 30 ]avec1 = avec[ 1 ] avec1 == avec[ 1 + false ]avec2 = avec[  2 ]avec2 == avec[ 1 + true ] ```As a start, recheck indexing expressions, be more sure they do what you want them to do.On Wednesday, November 16, 2016 at 1:36:57 PM UTC-5, Patrik Waldmann wrote:Hi,I'm an R user trying to learn Julia. I got hold of some code from the Knet package that I was playing around with. My goal is to set values to zero in a loop based on a logical expression, but I cannot figure out how the indexing works. Any help would be appreciated (the problem lies in w[1,(w.-(z))] = 0):using Knetpredict(w,x) = w*x .+ wlambda = 2z = Array{Float64}(1,13)loss(w,x,y) = sumabs2(y - predict(w,x)) / size(y,2)lossgradient = grad(loss)function train(w, data; lr=.1)    for (x,y) in data        dw = lossgradient(w, x, y)        z[:] = lr * lambda        w -= lr * dw        w -= lr * dw        w[1,(w.-(z))] = 0    end    return wendurl = "https://archive.ics.uci.edu/ml/machine-learning-databases/housing/housing.data"rawdata = readdlm(download(url))x = rawdata[:,1:13]'x = (x .- mean(x,2)) ./ std(x,2)y = rawdata[:,14:14]'w = Any[ 0.1*randn(1,13), 0 ]niter = 25lossest = zeros(niter)for i=1:niter; train(w, [(x,y)]); lossest[i]=loss(w,x,y); endBest regards,Patrik