By using AWS re:Post, you agree to the Terms of Use
/AWS X-Ray/

Questions tagged with AWS X-Ray

Sort by most recent
  • 1
  • 90 / page

Browse through the questions and answers listed below or filter and sort to narrow down your results.

OpenTelemetry with Elastic Beanstalk

Hi all I'd like to set up AWS Open Distro for OpenTelemetry for Java with Elasic Beanstalk (https://aws-otel.github.io/docs/getting-started/java-sdk/trace-auto-instr). I've added the Java agent (using `java -javaagent:path/to/aws-opentelemetry-agent.jar -jar myapp.jar`). Starting the application, it logs a line like this every 15 seconds: ``` [otel.javaagent 2022-06-08 19:21:03:940 +0000] [OkHttp http://localhost:4317/...] ERROR io.opentelemetry.exporter.internal.grpc.OkHttpGrpcExporter - Failed to export spans. The request could not be executed. Full error message: Failed to connect to localhost/127.0.0.1:4317 ``` So I guess the ADOT Collector (https://aws-otel.github.io/docs/getting-started/collector) isn't running. When I log into my instances (platform: Corretto 11 running on 64bit Amazon Linux 2/3.2.15), I can see the CloudWatch Agent under `/opt/aws/amazon-cloudwatch-agent` (version: 1.247350.0). It includes the Collector under `/opt/aws/amazon-cloudwatch-agent/cwagent-otel-collector`: ``` $ pwd /opt/aws/amazon-cloudwatch-agent $ ls LICENSE NOTICE RELEASE_NOTES THIRD-PARTY-LICENSES bin cwagent-otel-collector doc etc logs var $ ls cwagent-otel-collector/ etc logs var $ ls bin CWAGENT_VERSION amazon-cloudwatch-agent-config-wizard config-downloader cwagent-otel-collector amazon-cloudwatch-agent amazon-cloudwatch-agent-ctl config-translator start-amazon-cloudwatch-agent ``` I try to start the OTel Collector using `sudo bin/amazon-cloudwatch-agent-ctl -a start -m ec2 -o default`. It doesn't start, though. Here's the process status: ``` $ sudo bin/amazon-cloudwatch-agent-ctl -a status { "status": "running", "starttime": "2022-06-09T07:12:30+0000", "configstatus": "configured", "cwoc_status": "stopped", "cwoc_starttime": "", "cwoc_configstatus": "configured", "version": "1.247350.0" } ``` And the log: ``` $ tail cwagent-otel-collector/logs/cwagent-otel-collector.log 2022/06/09 07:09:20 E! User aoc does not exist: no matching entries in passwd file 2022/06/09 07:09:20 E! Failed to ChangeUser: no matching entries in passwd file 2022/06/09 07:09:20 no matching entries in passwd file ``` Has anyone gotten the OTel Collector working with Elastic Beanstalk? In tips for how to configure this? -martin
1
answers
0
votes
18
views
asked 20 days ago

Long response time for cloudfront misses

Need some help debugging this long response time I'm seeing from my cloudfront CDN for images that have not been cached. The outline of our setup is that we have a cloudfront cdn that responds with cached images when available. If no cached image is available, there's a lambda that pulls the requested image from s3 and resizes it using sharp.js, then sends the resized image as the response to the request. Cloudfront caches this image and then uses it for subsequent requests for the same image. The problem is that this handling takes 2-3s usually. You can see in [this](https://i.stack.imgur.com/uCt4W.png) screenshot. I'm only partially aware of the breakdown of those 2-3s. That screenshot is of logs from cloudfront, so that means the problem must lie somewhere within our cloudfront setup. The lambda itself takes 800-1300ms from start to finish, and that includes the time it takes to pull the image from s3, resize it, convert it to a buffer, then respond to the request. We already use the [http keepAlive](https://aws.amazon.com/blogs/networking-and-content-delivery/leveraging-external-data-in-lambdaedge/) optimization to reduce the latency of pulling the image from s3. However the lambda's running time is often 50% or less of the total response time, so that means there's another significant bottleneck elsewhere that I haven't discovered, and I'm not sure how to go about finding it. I've tried enabling AWS X-Ray to get more insights into the problem but our lambda is on Lambda@Edge, which doesn't support X-Ray. What else can I investigate and where else could I look?
4
answers
0
votes
30
views
asked 6 months ago

Disable Xray feature on local dev machine(Java Spring boot AOP project)

HI, I have a spring boot application (2.3.1) running in a AWS EC2 instance. I followed AWS documentation for Spring and AOP:https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-java-aop-spring.html I am able to see the traces and service map after I deploy on my AWS EC2 instance. But when I run my code on my local machine in dev environment(NOT on EC2), it throws an error. ================================================================================= 2020-08-19 10:22:55.655 \[pool-1-thread-1] INFO c.a.x.s.sampling.pollers.RulePoller -Polling sampling rules. 2020-08-19 10:22:56.670 \[pool-1-thread-1] ERROR c.a.x.s.sampling.pollers.RulePoller -Encountered error polling GetSamplingRules: com.amazonaws.SdkClientException: Unable to execute HTTP request: Connect to 127.0.0.1:2000 \[/127.0.0.1] failed: Connection refused: connect at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1175) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1121) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686) at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512) at com.amazonaws.services.xray.AWSXRayClient.doInvoke(AWSXRayClient.java:1257) at com.amazonaws.services.xray.AWSXRayClient.invoke(AWSXRayClient.java:1226) at com.amazonaws.services.xray.AWSXRayClient.invoke(AWSXRayClient.java:1215) at com.amazonaws.services.xray.AWSXRayClient.executeGetSamplingRules(AWSXRayClient.java:568) at com.amazonaws.services.xray.AWSXRayClient.getSamplingRules(AWSXRayClient.java:539) at com.amazonaws.xray.strategy.sampling.pollers.RulePoller.pollRule(RulePoller.java:65) at com.amazonaws.xray.strategy.sampling.pollers.RulePoller.lambda$start$0(RulePoller.java:46) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) ======================================================================================= Questions: 1.) We do not need the traces to be generated and sent to aws. How do we TURN OFF/DISABLE Xray tracing feature on our dev machines ?
1
answers
0
votes
41
views
asked 2 years ago

Re-configuring AWS X-Ray Daemon

Hi, I am deploying AWS X-Ray daemon on an EC2 Instance (AWS Linux AMI). I am doing this by following the instructions provided in <https://docs.aws.amazon.com/xray/latest/devguide/xray-daemon-ec2.html> which asks to have the following in UserData script: ``` #!/bin/bash curl https://s3.dualstack.us-east-2.amazonaws.com/aws-xray-assets.us-east-2/xray-daemon/aws-xray-daemon-3.x.rpm -o /home/ec2-user/xray.rpm yum install -y /home/ec2-user/xray.rpm ``` The above install launches the X-Ray Daemon and it listens to the default UDP bind address 127.0.0.1:2000 (Loopback address). I require to make the daemon process listen to the public network (0.0.0.0:3000) and also want changes to the region and logging configuration. <https://docs.aws.amazon.com/xray/latest/devguide/xray-daemon-configuration.html> describes how this can be achieved. But I am facing the following issues: 1.) I am facing problems killing the X-Ray daemon on instance start up through the User Data script. pkill is not working for xray. 2.) I am unsure of how to start X-Ray daemon in the user data script as it is not as straight forward as executing the X-ray command to run in background mode i.e ``` /usr/bin/xray -c "/home/ec2-user/xray_config/daemon_config.yaml" & ``` will not work. I am wondering if it would be possible to specify a configuration file during X-Ray daemon installation so that the default daemon process that kicks off after install is launched with the configuration parameters (UDP bind address etc) that I specify. Can anyone please help me with this? Regards, Edited by: shricloud on May 27, 2019 12:41 PM Edited by: shricloud on May 27, 2019 12:42 PM
1
answers
0
votes
21
views
asked 3 years ago
  • 1
  • 90 / page