Update:
After trying to figure this out, I have written a quick script in matlab to visualize my calibration. Turns out that with the same detections file I get widely different results also in my matlab visualization.
Background:
I use three cameras positioned on the same elevation around an animal to get visuals from all sides of the animal. So, you would expect when you plot camera positions that they are in some (near) isosceles triangle with vectors depicting the direction the camera is pointing to all pointing to the center of the triangle.

Here's an example of a (really) bad calibration:

And a good one:

Important to note! The bad one here had the lower(!) error of the two. Both based on the same detections file as stated before.

Here's the matlab code I quickly made if someone is interested (or can translate and include it in anipose for quick check after anipose calibrate command): --EDIT: Please see post below for better code.

(Question: is the calibration file in degrees? If not, then I'm missing something and the above plots and code is wrong!)

Ok, so I think it is in radians, not degrees. Using the code below I can now draw the calibration. Still, using the same detection I get widely different calibration files. Why is this?

[f,p] = uigetfile("*.toml");
S = readlines(strcat(p,f));
clear p f
C = convertStringsToChars(S);
for i=1:size(C,1)
    str = C{i,1};
    if ~isempty(str)
        mask(i,1) = str(1)=="[";
    end
end
mask = mask.*[1:size(mask,1)]';
mask(mask==0)=[];
% Now we have the starts of the camera definitions, with the last one being
% metadata (which we ignore for now)

%% Get all names:
name = C(find(contains(C,"name")),1);
name = cellfun(@(x) x(9:10),name,'UniformOutput',false);
%% Get the translations:
tr = C(find(contains(C,"translation")),1);
fl = regexp(tr,'[-+]?\d+\.?\d*','match');
clear tr
for i=1:size(fl,1)
    tr(i,:) = cellfun(@str2num,fl{i,1});
end
%% Get the rotations:
rot = C(find(contains(C,"rotation")),1);
fl = regexp(rot,'[-+]?\d+\.?\d*','match');
clear rot
for i=1:size(fl,1)
    rot(i,:) = cellfun(@str2num,fl{i,1});
end
rot = rot.*-1;
%% Creating the lines or quiver:
lineLength = 100;
for i=1:size(rot,1)
    l(i,:) = createQuiver([0 0 lineLength],rot(i,:));
    p(i,:) = l(i,:)+tr(i,:);
    % p(i,:) = createQuiver(tr(i,:),rot(i,:));
end


%% Visualization:
figure;
for i=1:size(tr,1)
    scatter3(tr(i,1),tr(i,2),tr(i,3),'filled');
    hold on;
    line([tr(i,1) p(i,1)],[tr(i,2) p(i,2)],[tr(i,3) p(i,3)]);
end
axis equal;
ylabel("Y");
xlabel("X");
zlabel("Z");

function [p] = createQuiver(origin,degrees)
    Rx = [1 0 0; 0 cos(degrees(1)) -1*sin(degrees(1));0 sin(degrees(1)) cos(degrees(1))];
    Ry = [cos(degrees(2)) 0 sin(degrees(2));0 1 0;-1*sin(degrees(2)) 0 cos(degrees(2))];
    Rz = [cos(degrees(3)) -1*sin(degrees(3)) 0;sin(degrees(3)) cos(degrees(3)) 0; 0 0 1];
    R = Rx*Ry*Rz;

    p = origin*R;
end

Hey I'm in a similar situation where I'm noticing that each round of calibration I do with my current detections pickle file produces errors ranging from 2.75-2000. I used your code to verify the translations and rotations but I was wondering if you were able to investigate further on if this is a bug of anipose that gives different calibration errors or if its just a bad setup?

Good to hear that it's not just me noticing this. I didn't explore this further because it would mean diving really deep into the code of Anipose (I've tried a bit, but couldn't really figure it out before getting to OpenCV code). What I think is going on is that there's some randomisation before starting the optimisation procedure to get the best calibration. My guess is that it is often ending up in some local minimum without a way to get out. I came to this idea when I noticed that long calibration files (Lots of detected boards, >1000 overlaps between camera pairs) suffer less from this it seems.

I'm now sort of working around it by making long calibration files, running calibrations multiple times and checking the result via Matlab, and really tightly fixing the cameras (in case a calibration fails one day I can use the previous, or next day's calibration).

Still, I would love to know what is going on to be sure that what we do is actually making full sense.

Yea I'm realizing our set up might need more adjustments than I thought. While our 2D DLC model runs great, our calibration detects hardly any overlap between two pairs of our cameras and alot of overlap between one pair. Your matlab code has been a huge help in validating if this result of calibration makes sense.

Im curious, for your matlab plot that is a good calibration, all cameras are pointing outwards of the center. Is that supposed to happen because intuitively shouldn't it be in the center? Also, for your videos, how do you ensure overlapping frames to take place?

Hello, I have noticed this issue over the years as people try anipose on different setups. It's been hard to debug. What makes the calibration algorithm work so well on some setups and so poorly on others? How can it generalize?

The randomness comes from the subsampling in the iterative bundle adjustment procedure. The relevant code is here: https://github.com/lambdaloop/aniposelib/blob/master/aniposelib/cameras.py#L700
This is the only way I have found calibration to actually arrive at a solution in some settings. It can help get out of local minima better than a complete bundle adjustement. Unfortunately, it ends up being stochastic and also doesn't fully solve the problem of local minima, as you see here.

I've had more time now so I've started to dive into this issue again, and can recommend a couple of things.

1. I've added some updates to anipose and aniposelib to improve the detections used for calibration.
   I recommend upgrading to the latest anipose (1.1.1) and aniposelib (0.6.1) and see if that helps.
   I have found especially using opencv-contrib-python > 4.7 helps a lot to provide better charuco board detections.
   Thanks to @philipqueen in this pull request for this update: Update opencv version to >= 4.7.* aniposelib#21

2. A trick that you can use within anipose is that, if you find a good calibration of your set up, you can use that as your initialization for calibration for the future. You just add a calibration_init variable in the [calibration] section of the config.toml. For instance:

[calibration]
# ...board parameters and stuff...
calibration_init = "calibration_init.toml"

In this case you would put a calibration_init.toml file that would be a copy of your best calibration.toml file for that setup.
Even if the cameras shifted a bit, it can be a great starting point for reliable calibration.

4. Last week in testing I found that optimizing the extrinsics only and then fine-tuning both extrinsics and intrinsics leads to much more robust and reliable calibration, especially in setups with fewer pairs of detections across cameras. This functionality is already possible within aniposelib, but I'll adapt it to anipose soon.

5. Regarding improved detections, I've also started to incorporate a better charuco detector using code from https://github.com/JunkyByte/deepcharuco/ into aniposelib. I think this should improve charuco detection in low light and slightly out of focus environments, as well as speeding up the detection (since it can use GPU).

@lambdaloop: This is really a great help! Many thanks for this reply!

1. I'm a bit hesitant to update Anipose now, since I'm in the middle of a project. When I have time I'll install in a new environment and go from there.
2. The trick with the initial parameters, is that general anipose, or only the new version you recommend here?

Re 4&5: Really great to maybe get these features in anipose as well! Many thanks for the great software and continuing support!

I'll start with the initial calibration since my cameras really should change not at all between sessions.

@ebtanana: Concerning the Matlab code, please use the second post I made, in the first one was a silly mistake.... The cameras should be pointing inward and make sense.