### sas >> BY processing in IML

Hi,

is there an equivalent way in IML to create a BY processing as we do usually in
data step ?

I have a matrix with some informations for several countries and I'd like to
treat each of them country by country.

Cordialement/
Best regards,

Stephane COLAS

**************************
SociDatametric

Notre site / Our site
http://www.datametric.fr
**************************

### sas >> BY processing in IML

What I usually do is to add an index column of integers (corresponding
to the levels of the by-values) in a data step prior to PROC IML. Then
in PROC IML I write a loop so that it loops over all the integers that
I just created. Inside the loop, you would operate only on those parts
of the data matrices that correspond to the rows that have the desired
integer index. To find the data rows that you want to select in each
iteration of the loop, you would use the IML LOC function.

Paige Miller
paige\dot\miller \at\ kodak\dot\com

>
> Date:    Wed, 29 Dec 2004 12:51:05 -0800
> From:    Dale McLerran < XXXX@XXXXX.COM >
> Subject: Re: Reducing a matrix in IML iteratively
>
> --- "Tonidandel, Scott" < XXXX@XXXXX.COM > wrote:
>
> > Dale,
> >
> > Thanks for the helpful suggestions. I think the macro is the way to
> > go
> > because this code is part of a larger simulation study that may have
> > as
> > many as 8 X-variables. I think I am going to go with the second macro
> > you suggested below but this led to a few additional questions (I
> > hope
> > you don't mind this additional imposition -- if so my apologies). The
> > first question has to do with the scope of my problem while the other
> > two stem more from my unfamiliarity with combining macros and IML.
> >
> > 1) I was originally thinking about reducing the entire correlation
> > matrix and then pulling the parts out of that reduced matrix that I
> > need (Sxx, Syy, Sxy). But, I liked the idea you presented to focus
> > on a piece of the original correlation matrix (Sxx in this case)
> > and reduce that.  But this leaves the additional step of me having
> > to reduce the Syy, and Sxy matrices as well. Is there an easy way
> > you would recommend doing that in the context of your macro?
>
> The Sxy matrix can be subset using the index vector.  Just
> employ Sxy[index,] and also Syx[,index].  I was under the
> impression that all Y variables would be included in the
> multivariate R-square computation, so that no subscripting would
> be needed for Syy.  If that is incorrect, then you will need
> additional loops which form an index for the columns of the
> multivariate response.
>
> >
> > 2) As I said earlier this is actually a piece of a larger simulation
> > where I am generating data with various numbers of predictor
> > variables and criterion variables so this will be nested within a
> > larger do loop.  In my larger program I have a scalar variable
> > called numpred which is indexed from 2 to 8 so the &columns
> > variable in your macro will need to take on these values. But,
> > the macro will not let me put numpred as an argument b/c it treats
> > it as text. I am not very familiar with combining macros and IML so
> > how would I make this numpred variable an argument in the macro?
>
> If I understand correctly, you have code something like the
> following:
>
>   do numpred=1 to 8;
>     do iter=1 to 1000;
>       <generate X data with numpred columns>
>       <generate Y data with numcrit columns>
>       <obtain multivariate R-square of Y with every combination of X>
>     end;
>   end;
>
>
> Now, employing the code that I posted yesterday, you would need
> to have NUMPRED inner do loops in order to obtain the multivariate
> R-square for every possible combination of the columns of X.
> But the number of loops cannot depend on the IML variable NUMPRED.
> Thus, the code that I sent yesterday will not work.
>
> Here is a different solution which will work for the problem
> described above.  We know that there are ((2**NUMPRED)-1)
> combinations of the predictor variables that must be considered.
> Now, if we loop from 1 to ((2**NUMPRED)-1) and convert the loop
> index value to binary, then we will have a set of values that
> are constructed as
>
>    do loop                      reversed
>     value     binary value    binary value
>       1         00000001        10000000
>       2         00000010        01000000
>       3         00000011        11000000
>       4         00000100        00100000
>      ...           ...             ...
>
>
> Note that I have represented the binary variable with eight digits
> which is the maximum value for the index variable NUMPRED.  Now,
> we can treat the columns of the reversed binary value as indicators
> for the columns of X that we should select for the indexed
> combination.  Below is code which constructs the reversed binary
> value, and from that constructs an index for the columns of X
> that should be selected.
>
>     proc iml;
>       do numpred=1 to 4;
>         do i=1 to ((2**numpred)-1);
>           Xcols = reverse(putn(i,"binary8."));
>           do j=1 to numpred;
>             xj=num(substr(Xcols,j,1));
>             if j=1 then index=xj*j;
>             else index=index || xj*j;
>           end;
>           index = loc(index);
>           print numpred Xcols index;
>         end;
>       end;
>     quit;
>
<stuff removed>

> HTH,
>
> Dale
>
> =====
> ---------------------------------------
> Dale McLerran
> Fred Hutchinson Cancer Research Center
> mailto:  XXXX@XXXXX.COM
> Ph:  (206) 667-2926
> Fax: (206) 667-5977
> ---------------------------------------

Dale,

I like the trick of looping over all possible combinations using
the binary numbers, however this does bring with it some overheads,
specifically the character string manipulation and then building up
the matrix index by repeated concatenation with itself.  Below
is some IML code for enumerating combinations that I have
taken from an old program that calculates multiple-case influence
diagnostics.  For me this has several advantages, firstly that
the subsets of different size are kept separate and secondly
that it should be a lot more efficient when the number of
combinations rises into the thousands.

proc iml;
reset noname;
/* IML module to loop over all combinations of k things from n.
Subsets are enumerated in increasing subset size from k1 to k2.
*/
start comblist(n,k1,k2);
do k=k1 to k2;
ncomb=round(exp(lgamma(n+1)-lgamma(k+1)-lgamma(n-k+1)));
index=1:k;
index[k]=k-1;
last=(n-k+1):n;
do s=1 to ncomb;
j=k;
do while (index[j]=last[j]);
j=j-1;
end;
index[j]=index[j]+1;
do i=j+1 to k;
index[i]=index[i-1]+1;
end;
print index [format=3.0];
end;
end;
finish;
run comblist(8,1,8);
quit;
run;

Kind regards,

Ian.

Ian Wakeling
Qi Statistics.


Does anyone know what happens to IMLPlus and IML Workshop with SAS 9.2?
Will version 2.1 still work? =20
=20
Harold L. Nelson, Jr. PhD
Sr. Forecast Coordinator
OFM Forecasting
Voice: (360)902-0603=20
=20


5. Proc IML

I have this matrix (named l) in proc iml;

8.5481086
3.1754262
2.4365849
2.1756817
2.1016003
2.0345899
2.0297729
2.0120973
1.9528005
1.8549354
1.7740571
1.5907812
1.3799565
1.2994083
1.2888679
1.2487197
1.208673
1.1776907
1.1335723
1.115597
1.105046
1.1009595
1.0816521
1.073351
1.0700454
1.06145
1.0457141
1.039487
1.0336203
1.0205739
1.0183625
1.006175
0.9997061
0.9943821
0.9934341
0.9869892
0.9812974
0.9779061
0.9616084
0.954517
0.9404594
0.9360798
0.9243392
0.9195776
0.9097169
0.8970483
0.8903767
0.883725
0.8702609
0.8668871
0.851951
0.8345694
0.8166014
0.8154286
0.7716379
0.7349595
0.698834
0.6883967
0.4687137
0.2321044
0.0031341
1.06E-15
3.176E-16
2.225E-16
1.741E-16
1.075E-16
3.982E-17
-1.9E-18
-3.09E-17
-4.3E-17
-8.95E-17
-1.23E-16
-1.43E-16
-1.62E-16
-2.27E-16
-3.34E-16
-4.96E-16
-8.29E-16

And I run this routine on it:

cond_num = l[1];
cond_indx = l;
do i=1 to nrows(l);
cond_indx[i] = cond_num / cond_indx[i];
end;
print cond_indx;

obtaining the matrix:

1
2.6919564
3.5082335
3.9289334
4.0674284
4.2013915
4.2113621
4.2483575
4.377359
4.6083053
4.8183953
5.3735288
6.1944768
6.5784626
6.6322614
6.8454985
7.0723087
7.2583649
7.5408587
7.6623627
7.7355231
7.7642351
7.9028262
7.9639453
7.9885479
8.0532373
8.1744227
8.2233918
8.2700668
8.3757857
8.3939743
8.4956481
8.5506219
8.596402
8.604606
8.6607924
8.7110274
8.7412366
8.8893864
8.9554282
9.0892906
9.1318159
9.247805
9.2956901
9.3964487
9.5291504
9.600553
9.6728152
9.8224667
9.8606948
10.033568
10.242538
10.467909
10.482964
11.077876
11.630721
12.231959
12.417417
18.237377
36.828727
2727.462
8.0652E15
2.6916E16
3.8412E16
4.9085E16
1.075E-16
3.982E-17
-1.9E-18
-3.09E-17
-4.3E-17
-8.95E-17
-1.23E-16
-1.43E-16
-1.62E-16
-2.27E-16
-3.34E-16
-4.96E-16
-8.29E-16

Why does the division go wrong starting at 1.075E-16 ?  Any help is appreciated.


Does anyone know how to calculate Cronbach's alpha in iml? Is there a
function?

Thanks,
Cristian