How can I scale / normalize data written in text files ?

Please I have training data input text file and output file (Attached). I need to scale the data between (0 to 1) before using them as training data in data mining then after training I would get an equation needs to be descaled (bringing it back as original scale).

 Many thanks

Answer 1

[Star Strider]

I used my own data rather than your text files because it is easier to test them.

I would do this:

d = randi([-99 99], 10, 1);                             % Create Data

extr = [min(d) max(d) max(d)-min(d)];                   % Extremes & Ranges
fwd_tran = (d - -sign(extr(1))*extr(1))/extr(3);        % Map To (0,1)
rev_tran = extr(3)*fwd_tran + -sign(extr(1))*extr(1);   % Map To Original

I believe this will do what you want. It is easy to create anonymous functions from ‘fwd_tran’ and ‘rev_tran’ if you need to:

fwd_tran = @(x) (x - -sign(extr(1))*extr(1))/extr(3);         % Map To (0,1)
rev_tran = @(x) extr(3)*x + -sign(extr(1))*extr(1);           % Map To Original

Test1 = fwd_tran(d);
Test2 = rev_tran(Q1);

Be sure to import your data and assign ‘extr’ before you use the functions.

[ND]

Many thanks Star,

Actually, that is right when I have data needs to be normalized then bringing back again to original. In my case I am using this data as training data to gives me a mathematical equation (model) and this model with parameters, exponents and variables needed to return as trained with original one.

Also, please how can I use this code with text files?

Best regards

[Star Strider]

My pleasure.

I just now edited my code to make it more robust. (Previously, it assumed all your data are both positive and negative. Now, it accounts for your data being either all positive or all negative as well. I tested it with all three options.)

If you created data in your model that were essentially those of the transformed data, then using ‘rev_tran’ on those would map them to your original data space.

You would have to read your text files into your workspace and then use my code. So my functions will work with your text file data, but only after you import them.

I would use the anonymous function implementations, since they ar easier to work with.

[ND]

OK, let me explaine a bit more,

The data come from finite element engine and used as training data into data mining software which gives me an equation returns to finite element again. What I need is just normalize this data and after getting model re-scale the model into original scale.

[Star Strider]

I do not understand what you are doing. My function ‘fwd_tran’ will take your ‘input’ data and map it to your ‘output’ data. My function ‘rev_tran’ will take your ‘output’ data and map it back to be the same as your ‘input’ data.

They will apply the same mapping to any data your function creates. The function you use depends on what your data are and what you want to map your data to. So if your function takes data on the interval (0,1) as its input, and produces data on the interval (0,1) as its output, you would use my ‘rev_tran’ function to map it back to the original space of your 'input.txt' file.

[ND]

After using the code I will get data normalized for input and output. use this data as training data which will give me equation like:

y = 10 * x^2 - 4* x^3 .... e.g

How can I now treat this equation as trained with normal data (scale parameters and exponents again?

Sorry for that but this is what I am looking for.

Many thanks

[Star Strider]

I do not understand how this relates to your original Question.

I will delete my Answer in a few minutes because it apparently is not providing the information you want.

[ND]

No sorry your answer is very close to what I need but in case of equation ( I explain that with the question) I do not know how can I treat the equation which I got it after training to the scaled data and put this equation in original scale?

Many thanks for all help

[Star Strider]

My pleasure.

To put the results of your equation back into the form (essentialy the ‘space’) of your original ‘input.txt’ data, use my ‘rev_tran’ function.

Just be sure to use your original data to calculate my ‘extr’ vector before you do anything else, and keep ‘extr’ in your workspace.