Cognito confirmDevice - Invalid device key given response

I am confirming a user's device after they complete MFA (serverside) to ensure that we are able for them to call refresh ( we need this due to device tracking on the user pool ). However, everything I try results in the same thing - Invalid device key given. We have it working fine on the client side without even trying due to aws-cognito-identity-js - but we also have a legacy login endpoint for older apps that we want to hook into the newer user pool and thus the refresh calls needs a deviceKey. I have been looking at the calls made by aws-cognito-identity-js which succeed in the browser - but fail when I copy as curl and run in the terminal with - yeah you got it Invalid device key given response. My code is below - accessToken & deviceKey come from the result of the response from the SMS_MFA challenge - has anyone got this working out of aws-cognito-identity-js?

            authHelper.generateHashDevice(deviceGroupKey, deviceKey, async (err) => {
                if (err) {
                    return reject(err);
                }

                const deviceSecretVerifierConfig = {
                    Salt: Buffer.from(
                        authHelper.getSaltDevices(),
                        'hex'
                    ).toString('base64'),
                    PasswordVerifier: Buffer.from(
                        authHelper.getVerifierDevices(),
                        'hex'
                    ).toString('base64'),
                };

                const confirmConfig = {
                    AccessToken: accessToken,
                    DeviceKey: deviceKey,
                    DeviceName: `${username}-${poolId}-${deviceKey}`,
                    DeviceSecretVerifierConfig: deviceSecretVerifierConfig,
                }
            
                await cognito.confirmDevice(confirmConfig);
            });