I cannot find ALB metric TargetConnectionErrorCount mentioned in the docs

As per documentation it says "Reporting criteria: There is a nonzero value" So that means, this metric will only emit data points if there is more than 0 data points. So right now there are no TargetConnectionError , that is why you dont see any metric. Once there are errors, you will see them.