From my understanding of the question, you want the model to be recomputed using all the inliers instead of only three. However, the algorithm has been implemented based on the following reference which uses only three inliers:
Xiao-Shan Gao, Xiao-Rong Hou, Jianliang Tang and Hang-Fei Cheng, "Complete solution classification for the perspective-three-point problem," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 8, pp. 930-943, Aug. 2003, doi: 10.1109/TPAMI.2003.1217599.
You can try to modify the code to use all inliers, but it is not recommended and may not produce the desired result.