You are trying to maximize utility, so you should minimize the negative of the utility. The following code runs without error for me:
[x,fval] = fmincon(@utility,[1 1],[],[],[],[],[0 0],[], @mycons)
function u = utility(x)
u = x(1)^(1/3)*x(2)^(2/3);
u = -u;
end
function s = cost(x)
s = 10*x(1) + 4*x(2);
end
function[c,ceq] = mycons(x)
c = cost(x) - 100 ;
ceq = [] ;
end
Alan Weiss
MATLAB mathematical toolbox documentation