MATLAB Answers

Common dependency management with source control

26 views (last 30 days)
William Warriner
William Warriner on 22 Jul 2019
Commented: William Warriner on 23 Jul 2019
I have two relatively large projects (A and B) that share a common dependency (C). A, B and C are all in their own separate git repositories. Now I want A to use code developed in B. There are multiple ways to handle this, none of which appear to be seamless.
If I use git submodules or subtrees, then adding A and B to the MATLAB path results in two copies of C being on the path. Due to function precedence the copies of functions of C in A are always called (A comes first alphabetically on the path). If the versions of C in A and B are out of sync, this can result in unexpected behavior in B.
I could solve this by putting C in private subfolders of A and B, so the expected version is called, but then many of the files in A and B must be in the top-level folder, which is not a very well organized solution. Additionally, my code depends on a number of FEX submissions, and it would be helpful to be able to organize those into their own "lib" folder to maintain a hard boundary between that code and my own code.
Alternatively, I could forego submodules, and manually add C separately. Then there is only one copy of C on the path. However, this results in a proliferation of repositories which must be independently cloned to access the functionality of a single project, especially with the number of FEX submissions.
Finally, I could just dump everything in a single repository, but then maintaining multiple copies of the dependencies, with respect to future projects, becomes nightmarish and violates DRY (don't repeat yourself).
Am I missing a more ideal and seamless way of handling this sort of dependency problem? I have googled a number of terms I thought might result in useful information, but haven't found anything aside the four options listed above.

  2 Comments

per isakson
per isakson on 23 Jul 2019
I'm trying to get my head around how to best use Matlab's git support and I've spent a couple of days reading and testing. E.g.
Comments so far
  • The Developer Zone blogs don't discuss the problem of dependencies.
  • There is no "ideal and seamless way of handling this sort of dependency problem", neither in Matlab nor elsewhere.
  • More context is needed to choose an approach. Single developer or a group. ...
"A, B and C are all in their own separate git repositories." fine, but what is the meaning of "the versions of C in A and B"? Is C a subtree or submodule of A and B, respectively? Why?
When working on A, is it likely that you will modify code in B and C? In my case that is likely.
William Warriner
William Warriner on 23 Jul 2019
Great questions. I work in academia developing software tools to support group research and for proof-of-concept for industry. I mostly do development work alone, but hope to bring in any other talent I can.
I allowed myself a couple of days to thrash around and experiment with this and realized that you are absolutely right, there is no best way to do this. I've tried subtrees, submodules, copy-paste, and even a package manager (https://www.mathworks.com/matlabcentral/fileexchange/54548-mpm) which was pretty neat but lacks recursive requirements checking (admittedly a pretty hard problem as I've learned). Even Microsoft has spent lots of time and money on a similar problem (https://devblogs.microsoft.com/bharry/the-largest-git-repo-on-the-planet/).
To answer your specific questions: I have tried both subtrees and submodules. Either way, there are two copies of C on the MATLAB path. By function precedence, only one is preferred. If for any reason those copies of C become desynchronized (say A gets a new version of C that would break B) then there can be unexpected or undesirable results (such as code in B failing to run). The same problem can occur whether C is a submodule or subtree of A and B.
I am consistently and frequently modifying code in B and C. As a result I'm moving to a monolithic repository. If I get additional people working on the code and can divide responsibilities, then it might be worth addressing the problem again.
I would still be interested to hear from you and others on what works and doesn't.
Thanks for your thoughts and help!

Sign in to comment.

Answers (0)

Sign in to answer this question.