For some reason the sharepoint people picker in Sharepoint Services 3.0 was not seeing domain users. This could be due to this sharepoint install not being installed by a user with access to the domain (i.e. a local user). To fix this use the “stsadm.exe” command:

stsadm -o setproperty -url http://<webapplicationurl> -pn “peoplepicker-searchadforests” -pv “domain:<fulldomainurl>,<username>,<password>”

where the username and password are for someone with access to the domain.

Although I have detailed another way of doing dimension reduction in Matlab I recently found the command “princomp” which does everything for you. The following code reads in .csv files from a directory and reduces them to a set number of dimensions (“OutputSize” in this case). This is a lot easier than doing it yourself with the eigenvectors etc:

function [output_args]=ReduceUsingPCA2(DirName,OutputSize)

files = dir(fullfile(DirName, ‘*.csv’));
for i=1:length(files)
% read files(i).name and process
FileName= [DirName '/' files(i).name];
% read in csv file from FileName and store in x
x = csvread(FileName);

% calculate PCs and project data onto principal components
[COEFF,SCORE] = princomp(x);

[infile, remain] = strtok(FileName,’/');
infile = strtok(remain,’.');
mkdir([num2str(OutputSize) 'PC']);
outputfilename = [num2str(OutputSize) 'PC' infile '_' num2str(OutputSize) 'PCs.csv'];
csvwrite(outputfilename,SCORE(:,1:OutputSize));
end
end

The important method is   [COEFF,SCORE] = princomp(x); which takes in your data “x” and stores its projection into PCA space in “SCORE” which I then output to csv. I still need to find out how to project back into normal space but I think it should be just as straightforward as this was. For more info on “princomp” type “help princomp” into matlab and have a look at the help files.

For the AIBO project we needed to stream webcam data from a webcam to an iframe on our website. We used an Axis IP camera but as soon as we had more than 2 people viewing the built in server crashed and the camera needed restarting. Although this problem didn’t happen every time, it happened enough that we decided to go with a webcam and SimpleCam from Dead Pirate Software, which I have used before.

The setup is easy, just make sure your webcam is installed, run SimpleCam and make sure the server port is set. To view the feed just visit the ip and port of the pc with the webcam attached. There are more instructions with pictures on the Dead Pirate site.

I needed to run 200 odd batch files sequentially in the background for a speech classifier. The code to do this involved writing a batch file in code to include the commands i wanted the HTK to run, probably not the best way of doing this but HTK is funny with file paths with spaces etc and this was the quickest way.

Process hvite = new Process();
hvite.EnableRaisingEvents = false;
hvite.StartInfo.FileName = “temp.bat”;
hvite.StartInfo.CreateNoWindow = true;
hvite.StartInfo.WindowStyle = ProcessWindowStyle.Hidden;
hvite.Start();
hvite.WaitForExit();

This information is out of date really, I have a much easier method here that does away with doing everything yourself.

I used Matlab to reduce the number of dimensions in my gesture data. After a bit of experimentation with different numbers of dimensions I found I could reduce the number of dimensions by half using PCA and still get quite low errors between the original data and the reduced dimension reconstructed data. Some gesturers made such consistent movements I could use just 2 dimensions to describe almost their entire range of motion.

The method is relatively clear in Matlab, although I am still a bit unsure of the multiple transforms made in the following code. I think I may have performed a few too many, but at least it works! The code “ReduceUsingPCA.m” takes in a directory to perform the conversion on and the number of output dimensions you require. So to convert every .csv in “c:\input” to 20 dimensional data you run it as “ReduceUsingPCA(“c:\input”,20) in Matlab.

% FileName is the name of the file to work on, OutputSize is no. of
% dimensions to output after PCA
function [output_args]=ReduceUsingPCA(DirName,OutputSize)

files = dir(fullfile(DirName, ‘*.csv’));

for i=1:length(files)
% read files(i).name and process
FileName= [DirName '/' files(i).name];
% read in csv file from FileName and store in x

x = csvread(FileName);

[Rows, Columns] = size(x);  % find size of input matrix
m=mean(x);                  % find mean of input matrix
y=x-ones(size(x,1),1)*m;    % normalise by subtracting mean
c=cov(y);                   % find covariance matrix
[V,D]=eig(c);               % find eigenvectors (V) and eigenvalues (D) of covariance matrix
[D,idx] = sort(diag(D));    % sort eigenvalues in descending order by first diagonalising eigenvalue matrix, idx stores order to use when ordering eigenvectors
D = D(end:-1:1)’;
V = V(:,idx(end:-1:1));     % put eigenvectors in order to correspond with eigenvalues
V2d=V(:,1:OutputSize);        % (significant Principal Components we use, OutputSize is input variable)
prefinal=V2d’*y’;
final=prefinal’;            % final is normalised data projected onto eigenspace

[infile, remain] = strtok(FileName,’/');
infile = strtok(remain,’.');
mkdir([num2str(OutputSize) 'PC']);
outputfilename = [num2str(OutputSize) 'PC' infile '_' num2str(OutputSize) 'PCs.csv'];

csvwrite(outputfilename,final);
end
end

The files are saved in the same directory as the input data, eg: “filename20PCs.csv”

The output of my Principal Component Analysis in Matlab to reduce the dimensionality of gesture data is in the comma separated variable format. 57 dimension data goes in, X dimension data comes out in standard csv format. It is better to remove unnecessary information from the gesture data as it only makes the recognition of gesture (and intent, in my case) more difficult.

The problem is that the HTK, which I am going to use to perform recognition, doesn’t natively accept csv data so you have to convert to the HTK binary format parameter files. I chose to do this in c# as I’m familiar with it, but I stumbled across a few problems relating to the conversion between big-endian and little-endian binary data. HTK reads data in the opposite way to my PC (although I’m sure I read on their website somewhere that there is automatic detection for this).

The following code is pretty rough around the edges as it includes a lot of stuff to help me debug it. The program reads a directory and converts all *.csv files into HTK format binary files by reading in the data as floats, converting to bytes, writing a header and then writing the data to a binary file *.csv.bin.

static void Main(string[] args)
{
string dir = @”G:\PHD Nov 09\# programs\matlab work\test”;
DirectoryInfo di = new DirectoryInfo(dir);

FileInfo[] rgFiles = di.GetFiles(“*.csv”);
foreach (FileInfo fi in rgFiles)
{
using (TextReader tr = new StreamReader(fi.FullName))
{
string data = tr.ReadToEnd();
System.Text.ASCIIEncoding encoding=new System.Text.ASCIIEncoding();
byte[] byteArray = encoding.GetBytes(data);

string newdata = data.Replace(‘\n’,’ ‘);
string[] plit = newdata.Trim().Split(‘ ‘);

int samples = plit.Length;
int itemspersample = plit[0].Split(‘,’).Length;

// now create binary data, each sample (part of a line in the file)
// has to be converted from a float to a 4 byte array and then joined to make one
// large binary file

byte[] bytedata = new byte[samples * itemspersample * 4];

for (int i = 0; i < samples; i++)
{
for (int j = 0; j < itemspersample; j++)
{
string dd = plit[i].Split(‘,’)[j];
float f = (float)Convert.ToDouble(plit[i].Split(‘,’)[j]);

byte[] temp = new byte[4];
temp = BitConverter.GetBytes(f);

bytedata[(i * itemspersample * 4) + (j * 4)] = temp[3];
bytedata[(i * itemspersample * 4) + (j * 4) + 1] = temp[2];
bytedata[(i * itemspersample * 4) + (j * 4) + 2] = temp[1];
bytedata[(i * itemspersample * 4) + (j * 4) + 3] = temp[0];
}
}

// now create HTK header 12 bytes long
byte[] nSamples = BitConverter.GetBytes(samples);
byte[] sampPeriod = BitConverter.GetBytes(100000);
byte[] sampSize = BitConverter.GetBytes(Convert.ToInt16(itemspersample * 4));
byte[] parmKind = BitConverter.GetBytes(Convert.ToInt16(9));

using (BinaryWriter bw = new BinaryWriter(File.Open(fi.FullName + “.bin”, FileMode.Create)))
{
Array.Reverse(nSamples);
Array.Reverse(sampPeriod);
Array.Reverse(sampSize);
Array.Reverse(parmKind);
bw.Write(nSamples);
bw.Write(sampPeriod);
bw.Write(sampSize);
bw.Write(parmKind);
bw.Write(bytedata);
}
}
}

To check it works you run HList, with no config file required as the header explains to HTK everything it needs to know about the data:

G:\PHD Nov 09\# programs\matlab work\test>hlist -h EO412_10PCs.csv.bin
————————- Source: EO412_10PCs.csv.bin ————————
Sample Bytes:  40       Sample Kind:   USER
Num Comps:     10       Sample Period: 10000.0 us
Num Samples:   5        File Format:   HTK
—————————— Samples: 0->-1 ——————————
0:    1838.200 308.910-262.970 401.920 -66.737-499.370 305.260-260.250 -91.974  28.171
1:    1837.700 308.630-263.340 400.810 -67.144-499.920 305.280-260.060 -92.174  27.584
2:    1837.000 308.360-263.750 399.940 -67.870-500.510 305.540-259.960 -91.964  26.922
3:    1836.500 308.160-264.000 398.500 -68.003-501.230 305.790-259.980 -92.138  26.342
4:    1837.000 308.360-263.750 399.940 -67.870-500.510 305.540-259.960 -91.964  26.922
———————————– END ———————————–

This post is to be updated as I dig out more of my original notes and bookmarks. It was written for an overview document to be handed to the next person handling the Oracle project:

The Oracle ADF development guide gives a good overview of the entire Application Development Framework as provided by Oracle and is targeted at Oracle 10g. There is a short presentation on ADF available.
Migration to Oracle from Access is best done using the ODBC connection to Oracle and standard SQL commands. There are other alternatives but most migration tools didn’t work properly or were expensive. Some of the tools tried include Oracle SQL Developer, EMS Data import, Apatar, Oracle Migration Workbench (see this forum thread).

The installation process of Oracle on Solaris took some time to get to work but despite a few typos this guide is the best source of information on Oracle installation. More information is available here but this is a slightly different installation method. My blog at http://jamesrossiter.wordpress.com has some of the steps I took to install Oracle as well as a few of the pitfalls in the documentation to watch out for.
There is some slightly outdated info on Oracle’s built in http server here and here which helped explain a few of the different components Oracle needs for http.

The Oracle web server of choice is OC4J, Oracle Containers for Java. The most useful tutorials for working with JDeveloper and OC4J were on deployment of applications as well as this, the most important tutorial in this list on JSF, EJB and Oracle’s preferred way of writing web apps with JDeveloper. OC4J is based on Apache and must be installed separately to Oracle. A cut down old version of OC4J is included with Oracle 10g but this is not to be used to develop apps. A lot of the documentation is confusing on this but after checking the forums it appears you need to install a standalone OC4J instance or install the Oracle Application Server. I installed the standalone OC4J as it seems much easier to use and the documentation I found is mainly focused on the standalone version.

I had a look at Oracle htmlDB, which has been updated and is now called Apex (Oracle Application Express). This is supposed to be a very quick way of prototyping database driven applications, see this unofficial overview. There is an unofficial wiki for Apex. I got Apex to work but the amount of flexibility provided was not enough and I was concerned that we would outgrow the abilities of Apex when writing custom apps.

The best way to write online apps looks to be the ADF way of doing things with Jdeveloper, Oracle 10g (or 11) and OC4J. For this you need to know Enterprise Java Beans, Java Server Faces and how to use Jdeveloper to deploy your applications to OC4J. I had problems deploying applications to my Solaris install of OC4J using Jdeveloper but have tested several small apps based on the tutorial (in bold above) using the built in OC4J server in Jdeveloper. My forum posts with the problems I faced following the deployment tutorial are here and here but I still didn’t manage to solve the problem. There is a third party forum devoted to developing for Oracle.

The key to Jdeveloper web application creation is Master-Detail relationships, made possible through foreign key relationships in the Oracle database and automatic object creation using Java Beans. Master-Detail relationships can be easily visualised using the templates available in Oracle ADF Server Faces (a more Oracle focused version of Java Server Faces) and there are plenty of tutorials available from Oracle like those in the official documentation.

I used Skybound GeckoFX to embed firefox in my c# windows forms. Goodbye terrible .net webbrowser component!

I have tried several different techniques using loads of different software to rip chapters of a DVD directly to wmv. It seems that it has to be done in several stages, with a final stage of AVI to WMV as windows media encoder seems to crash every time I give it a .vob or .mpg file to convert.

I tried using dvd decrypter to create .vobs for each chapter in IFO mode but ended up with sound de-synchronisation despite the framerate etc being set correctly. After trying a lot of other programs (all open source) such as dvdx (crashed creating .wmv with my required resolution of 720×576) I ripped the dvd using Nero to one big .mpg then used Sony Vegas to cut it into chapters. I saved these chapters out as .mpg hoping to convert directly to .wmv using Windows Media Encoder 9 but the encoder would crash opening them.

Eventually I gave up and saved the mpg files as raw avis using virtualdub with the mpeg 2 plugin from fccHandler. I then used a great batch script from Tech Recipes to convert these avi files to the wmv format I was after.

So steps I used were: DVD > Nero conversion to one big mpeg > Sony Vegas manual chapter splitting and saving as mpeg > virtualdub mpeg2 conversion to raw avi > batch conversion of avi to wmv.

There must be a better way of doing this but every known way of doing it either didnt allow me to save out the chapters or caused audio synchronisation errors. I know Nero can see the chapters but why wont it let you rip them?

One of the biggest things missing from DotNetNuke is a robust versioning system to allow you to roll back changes and mistakes made by content editors. I’m not the only one that thinks so either as can be seen from the Roadmap page of the official site. A link on this page led me to Effority.net where I found this free module for workflow and versioning of Text/HTML modules. It’s not quite content versioning for entire pages or sites but is a good start and will hopefully allow us to correct mistakes (and entire Text/HTML content deletions) by some of the content providers.