Model-Free Iterative Learning Control of SISO System

Since R2024a

This example implements model-free iterative learning control (ILC) to improve closed-loop trajectory tracking performance of a single-input, single-output (SISO) system.

Iterative Learning Control Basics

Iterative learning control is an improvement in run-to-run control. It uses frequent measurements in the form of the error trajectory from the previous batch run to update the control signal for the subsequent batch run. The focus of ILC has been on improving the performance of systems that execute repeated operations, starting at the same initial operating condition. This focus includes many practical industrial systems in manufacturing, robotics, and chemical processing, where mass production on an assembly line requires repetition.

To understand how ILC works, consider a generic discrete SISO plant in the state space form.

$$x_{\mathit{k}}\left(\mathit{t}+1\right)={\mathrm{Ax}}_{\mathit{k}}\left(t\right)+{\mathrm{Bu}}_{\mathit{k}}\left(t\right),{\mathit{x}}_{\mathit{k}}\left(0\right)=x_{\mathit{o}}$$

$${\mathit{y}}_{\mathit{k}}\left(t\right)={\mathrm{Cx}}_{\mathit{k}}\left(t\right)$$

Here, $\mathit{t}\in \left[0,\mathit{N}\right]$ is the time index across the operation duration and $\mathit{k}$ is the iteration number during the ILC learning process.

You can represent the input-output relationship in the following lifted form.

$$\begin{array}{l}{\mathit{y}}_{\mathit{k}}\left(1\right)=C\left({\mathrm{Ax}}_{\mathit{k}}\right(0)+{\mathrm{Bu}}_{\mathit{k}}(0\left)\right)\\ {\mathit{y}}_{\mathit{k}}\left(2\right)=C\left({\mathrm{Ax}}_{\mathit{k}}\right(1)+{\mathrm{Bu}}_{\mathit{k}}(1\left)\right)=C\left(A\right({\mathrm{Ax}}_{\mathit{k}}\left(0\right)+{\mathrm{Bu}}_{\mathit{k}}\left(0\right))+{\mathrm{Bu}}_{\mathit{k}}(1\left)\right)={\mathrm{CA}}^2{\mathit{x}}_{\mathit{k}}\left(0\right)+{\mathrm{CABu}}_{\mathit{k}}\left(0\right)+{\mathrm{CBu}}_{\mathit{k}}\left(1\right)\\ \cdots \end{array}$$

As a result, the input-output relationship for all time $\mathit{t}\in \left[0,\mathit{N}\right]$ between vectors ${\mathit{u}}_{\mathit{k}}$ and $y_{\mathit{k}}$ can compactly written as:

$$y_{\mathit{k}}={\mathrm{Gu}}_{\mathit{k}}+d$$

where

$$\mathit{G}=\left[\begin{array}{cccc}\mathrm{CB}& 0& 0& \cdots \\ \mathrm{CAB}& \mathrm{CB}& \cdots & \\ ⋮& ⋮& ⋮& ⋮\\ {\mathrm{CA}}^{\mathit{N}-1}\mathit{B}& {\mathrm{CA}}^{\mathit{N}-2}\mathit{B}& \cdots & \cdots \end{array}\right]\mathrm{and}\mathit{d}=\left[\begin{array}{cccc}{\mathrm{CAx}}_0& {\mathrm{CA}}^2{\mathit{x}}_0& \cdots & {\mathrm{CA}}^{\mathit{N}}{\mathit{x}}_0\end{array}\right]$$

As a simple example, assume plant $G$ is stable and ILC is the only source that provides control action.

A general ILC update law is of the following form [1, 2].

$$u_{\mathit{k}+1}\left(t\right)=Q\left(q\right)\left[u_{\mathit{k}}\right(t)+L(q\left)e_{\mathit{k}}\right(\mathit{t}\left)\right]$$

Here, $Q$ is a low-pass filter that removes control chatter and $\mathit{L}$ is a learning function. There are multiple ways to design the learning function $\mathit{L}$. The simplest method is called model-free ILC (or Arimoto ILC) [3]. Model-free ILC does not require prior knowledge of the system dynamics and uses proportional error feedback to update the control history. In other words, the model-free ILC update law is:

$$u_{\mathit{k}+1}\left(t\right)=Q\left(q\right)\left[u_{\mathit{k}}\right(t)+\alpha e_{\mathit{k}}(\mathit{t}\left)\right]$$

where $\alpha$ is referred as the ILC gain.

With a properly chosen $\alpha$ value, the nominal asymptotic convergence of the tracking error to zero can be achieved.

Assume the plant is defined as $y_{\mathit{k}}={\mathrm{Gu}}_{\mathit{k}}+d$ and the tracking error is ${\mathit{e}}_{\mathit{k}+1}=\mathit{r}-{\mathit{y}}_{\mathit{k}+1}$. Then, using the model-free ILC update law, the error term is:

$$\begin{array}{l}{\mathit{e}}_{\mathit{k}+1}=\mathit{r}-\left({\mathrm{Gu}}_{\mathit{k}+\mathrm{1}}+\mathit{d}\right)\\ =\mathit{r}-\left(\mathit{G}\left({\mathit{u}}_{\mathit{k}}+\alpha {\mathit{e}}_{\mathit{k}}\right)+\mathit{d}\right)\\ =\mathit{r}-\left({\mathrm{Gu}}_{\mathit{k}}+\mathit{d}\right)-\alpha {\mathrm{Ge}}_{\mathit{k}}\\ =\left(\mathit{I}-\alpha \mathit{G}\right){\mathit{e}}_{\mathit{k}}\end{array}$$

Therefore, the tracking error asymptotically converges to zero if and only if $|\mathit{I}-\alpha \mathit{G}|<1$.

In the ILC tuning process, you can start with a small $\alpha$ and gradually increase its value for faster convergence until the ILC learning process becomes unstable.

Closed-Loop Model-Free Iterative Learning Control

In industrial practices, the plant is most likely already under some type of feedback or feedforward control to achieve stabilization and baseline tracking and regulation. When the operation is repeating, you can add ILC on top of the baseline controller to improve control performance.

In this example, you simulate a system that uses a model-free ILC controller to assist an existing PI controller.

Open the Simulink® model.

mdl = "scdmodelfreeilc";
open_system(mdl)

Plant and PI Controller

The plant, implemented in the Plant block, is the following linear, discrete-time, second-order system.

Ts = 0.01;
G = ss([-0.7 -0.012;1 0],[1;0],[1 0],0,Ts);

Design a baseline PI controller using the pidtune function.

c = pidtune(G,"pi");

Implement the controller using the PID block.

ILC Mode Generator

The ILC learning process involves two modes: learning and reset. In learning mode, ILC outputs ${\mathit{u}}_{\mathit{k}}$ at the desired time points and measures the error between the reference ${\mathit{r}}_{\mathit{k}}$ and output ${\mathit{y}}_{\mathit{k}}$. At the end of learning mode, ILC calculates the new control sequence ${\mathit{u}}_{\mathit{k}+1}$ to be used in the next iteration. In reset mode, the plant returns to the initial operation condition, driven by the existing control mechanism. The ILC controller output is 0 during reset mode. The ILC Mode Generator subsystem defines the length of each mode and provides a signal that switches between learning and reset modes, which downstream modules such as Reference Trajectory Generator and ILC can use.

Reference Trajectory Generator

In this example, you use a generic reference signal. The Reference Trajectory Generator subsystem outputs the desired reference signal during the learning mode, starting at zero. Then, in reset mode, it generates a reference of zero, which allows the PI controller to bring the plant back to the initial operating point.

Model-Free ILC Controller

The model-free ILC system contains two components:

• The ModelFreeILC block implements the main ILC update law using a MATLAB System block.

• The Low Pass Filter block implements filter $Q$.

You can configure the ModelFreeILC block using the following parameters.

• ILC gain: model-free ILC gain $\alpha$.

• Duration: duration of the repetitive operation.

• Time step (Ts): ILC sample time (match to the low-pass filter sample time)

This block updates the control history based on stored error history from the previous iteration.

Simulation With and Without ILC

To illustrate the benefit of ILC, simulate the system both with and without ILC control enabled. You can then compare the closed-loop control performance before and after ILC is used.

Define ILC controller parameters.

Tfinal = 8;
alpha = 3;

Define the gain for the low-pass filter.

filter_gain = [0.03921 -0.9608];

Configure the model to use only PI control.

set_param(mdl + "/ILC_On","sw","0");

Simulate the system for 128 seconds, which covers eight repetitive operations.

simout_pid = sim(mdl);

Plot the reference tracking results for the PI controller.

% Reference signal 
r = squeeze(simout_pid.logsout{4}.Values.Data);
% System output y_k
y_pid = squeeze(simout_pid.logsout{3}.Values.Data);
% Mode signal
mode = squeeze(simout_pid.logsout{1}.Values.Data);

figure
plot(simout_pid.logsout{4}.Values.Time,r)
hold on
plot(simout_pid.logsout{1}.Values.Time,mode,"--")
plot(simout_pid.logsout{3}.Values.Time,y_pid)
xlabel("Time")
ylabel("y")
legend("ref","Mode","yPID")
title("PID controller, Trajectory tracking")
hold off

The PID controller achieves the same performance for each of the repetitive tasks.

Configure the model to use both ILC and PID.

set_param(mdl + "/ILC_On","sw","1");

Simulate the model for eight repetitive operations.

simout_ilc = sim(mdl);

Plot the reference tracking results for the combined control system.

% Reference signal 
r = squeeze(simout_ilc.logsout{4}.Values.Data);
% System output y_k
y_ilc = squeeze(simout_ilc.logsout{3}.Values.Data);
figure(2)
plot(simout_ilc.logsout{4}.Values.Time, r)
hold on
plot(simout_pid.logsout{1}.Values.Time, mode,"--")
plot(simout_ilc.logsout{3}.Values.Time, y_ilc)
xlabel("Time")
ylabel("y")
legend("ref","Mode","yILC")
title("ILC controller, Trajectory tracking")
hold off

For the first operation, the control performance matches that of the PI controller case. As the iterations progress, the ILC controller improves the reference tacking performance.

The ILC control action, which compensates PI control, for the given repetitive reference tracking is shown in the following plot.

uILC = squeeze(simout_ilc.logsout{2}.Values.Data);
figure
plot(simout_ilc.logsout{2}.Values.Time,uILC)
xlabel("Time")
ylabel("ILC action")
title("ILC Control Effort")

References

[1] Bristow, Douglas A., Marina Tharayil, and Andrew G. Alleyne. “A Survey of Iterative Learning Control.” IEEE Control Systems 26, no. 3 (June 2006): 96–114. https://doi.org/10.1109/MCS.2006.1636313.

[2] Gunnarsson, Svante and Mikael Norrlöf. “A Short Introduction to Iterative Learning Control.” (1997).

[3] Hätönen, J., T.J. Harte, D.H. Owens, J. Ratcliffe, P. Lewin, and E. Rogers. “Discrete-Time Arimoto ILC-Algorithm Revisited.” IFAC Proceedings Volumes 37, no. 12 (August 2004): 541–46. https://doi.org/10.1016/S1474-6670(17)31525-2.

[4] Lee, Jay H., Kwang S. Lee, and Won C. Kim. “Model-Based Iterative Learning Control with a Quadratic Criterion for Time-Varying Linear Systems.” Automatica 36, no. 5 (May 2000): 641–57. https://doi.org/10.1016/S0005-1098(99)00194-6.

[5] Harte, T. J., J. Hätönen, and D. H. Owens *. “Discrete-Time Inverse Model-Based Iterative Learning Control: Stability, Monotonicity and Robustness.” International Journal of Control 78, no. 8 (May 20, 2005): 577–86. https://doi.org/10.1080/00207170500111606.