Why can an instance in a target group not reach itself via NLB?

I assume you have an internal load balancer with targets registered by instance ID. This is because of hair pinning / loop back issues. Internal network load balancers do not support hairpinning or loopback. When you register targets by instance ID, the source IP addresses of clients are preserved. If an instance is a client of an internal load balancer that it's registered with by instance ID, the connection succeeds only if the request is routed to a different instance. Otherwise, the source and destination IP addresses are the same and the connection times out.

Network Load Balancers preserve the source IP, so both the source and destination of the arriving packet are the private IP address of the target. Then, the host operating system sees the packet as invalid and fails to send response traffic, which causes the connection to fail. To correct this type of Network Load Balancer connection failure, use an IP target type. With an IP target type, the target sees the IP address of the Network Load Balancer. Because the source and destination are unique, the connection succeeds.

To summarize, if you have the requirement of accessing same target through NLB, use IP based target group.

Additional details are here - https://aws.amazon.com/premiumsupport/knowledge-center/target-connection-fails-load-balancer/ , https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-troubleshooting.html#loopback-timeout