Clear Filters
Clear Filters

What is ICMFrequency? and How do you specify it?

1 view (last 30 days)
I came across with an issue while fitting ecdf function for a large dataset (>700 ). When I specify the NameValue parameter "ICMFrequency" as anything greater than 15 it works almost instantly. The default value is supposed to be 10 and when I set it to default, it sits there for ever. I would like to know some clarification regarding this parameter. I have searched everywhere including google searches but no luck so far. Thanks in advance.

Accepted Answer

Liam Walsh
Liam Walsh on 23 May 2022
Hi Pappu,
To find the ecdf for interval-censored data, ecdf uses the EMICM algorithm as laid out in:
Anderson-Bergman, Clifford. "An Efficient Implementation of the EMICM Algorithm for the Interval Censored NPMLE." Journal of Computational and Graphical Statistics 26, no. 2 (April 3, 2017): 463–67.
This algorithm is iterative, and works by alternating until the ecdf is found between two different procedures - an expectation-maximization procedure (EM), and an iterative convex minorant procedure (ICM). The ICMFrequency Name-Value pair controls the frequency of the ICM step. By default, the algorithm will compute the ICM every 10th iteration, and the EM step every other iteration. Changing the ICMFrequency paramter changes how often the ICM algorithm is run. For more information, please see this doc page:
Are you willing to share your data? I'd like to investigate a bit further why you are seeing this behavior with the ICMFrequency with your data.
  4 Comments
Liam Walsh
Liam Walsh on 7 Jun 2022
Hi Pappu,
My apologies for the delay! I have investigated the issue further, and have found the source of the problem. Thank you again for sending along your data.
I believe you recently reported an issue with the ecdf function as it relates to double censoring - the root issue there is the same reason why setting ICMFrequency to 10 doesn't converge, but setting it to 15 does. Important to note though - the answer you are getting from ecdf when ICMFrequency is the correct answer.
A fix is in the works for both issues. Based on how the tech support systems works, you should receive an email when the first issue you reported with double censoring is fixed, and when you receive that email, know that this issue will also be fixed.
I am sorry for the inconvenience with using this function.
Best,
Liam
Pappu Murthy
Pappu Murthy on 7 Jun 2022
Thanks much for the update. I will wait for the updated function.

Sign in to comment.

More Answers (0)

Categories

Find more on Creating and Concatenating Matrices in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!