Questions tagged with Amazon Redshift

Hello, I'm running RLS to see how it works (aws.amazon.com/blogs/big-data/achieve-fine-grained-data-security-with-row-level-access-control-in-amazon-redshift/) I am successfully able to run Example 1 in the documentation but when I run example 2; it won't return any results. Example 2 uses session context variables. I'm able to create a user (external_user), grant role external to the user. Create and attach policy (see_only_own_customer_rows) to customer table and role external. I also login with the external_user and set the set_config variable with value as below- select set_config('app.customer_id', 'AAAAAAAAJNGEGCBA', FALSE); After running the select * query below I don't get any results. select * from report.customer Any ideas what could be the issue? Looks like everything is running fine but don't get the results on select * Any help would be really appreciatedlg...

I have a serverless database set up in Redshift, created a table, and am now trying to load that table from a .csv file I have uploaded to an S3 bucket. When I created the table I set the primary key as an identity key as follows: customerid integer NOT NULL identity(0,1) When I load the table using the COPY query, the key increments by 128 starting with 64 rather than by 1 starting at 1. For example, my customerID field has values of 64, 192, 320, 448, etc. I've read numerous articles that this is due to compression and parallelism. I've tried including the "COMPUPDATE off" command as part of my COPY query but that did not change the results. I've truncated my table each time before I've tried to reload it to reset the seed. How can I load a table and have the identity key start with 1 and increment by 1?lg...

I am loading the data from Amazon RDS(mysql database) to Redshift using AWS Glue ETL and data catolgue. But I can't figure out how to do incremental loading(upsert)? Is there a way to create a filter/parameter on date while reading from source database to load only yesterday's data without using bookmarks?lg...

Redshift Serverless Charging The Charge of Compute usage on Frankfurt region for Redshift Serverless is not matching the log on the sys_serverless_usage . I have charged by 758 RPU -Hour on the month of sep till 28th , while the below query for the same period returned 76 RPUHour ? select date_trunc('day',start_time) usage_date, sum(compute_seconds) com_sec, sum(((compute_seconds) ) )/ (60 * 60 ) total_consumer_hour, (sum(((compute_seconds) ) )/ (60 * 60 ) ) * (0.45) compute_cost from sys_serverless_usage group by date_trunc('day',start_time) I believe the issue on the charging is dividing the compute seconds by (60 times 6) rather than (60 times60), this is also happen when setting alert on RPU-USAGE is also alert on the number i post (example 20 RPU-H daily) it alert on 2 RPU-H.lg...

Hello, There are several people in our group that create their own redshift database in a provisioned cluster. I created a database in that cluster as well. Now I want to define roles in database and attach RLS (Row level security) policy to the roles I create. After creating first role I got error that role already exists. 1) How do I see what roles are already created in a Redshift database? 2) Can I see info about the roles - created/updated date and any policies attached to the roles in Redshift? 3) Is it possible to use same roles across different Redshift databases in same cluster and databases across different clusters?lg...

Hi Is it possible to connect AWS managed MySQL to be connected as external schema in Amazon Redshift cluster, which is in different account. Please let know if there is any series of steps which needs to be followed.lg...

Hi folks , got the below error while running query in athena editor after triggering crawler SYNTAX_ERROR: line 1:8: SELECT * not allowed from relation that has no columns This query ran against the "demodb" database, unless qualified by the query **Note** : the table got populated in the workgroup but when I try to preview table it should run the query but got the syntax error , Any help would be great , Thankslg...

I want to enter into digital game distribution services, similar to STEAM. I am new to all this. Need help and suggestions on how can I develop a steam-like platform where 1. There will be 2 interfaces - 1 for gamers, 1 for the game developer 2. Game codes to be stored in the cloud 3. User authentication will be needed 4. Game license management will be needed Thankslg...

Hi all, Our Redshift cluster (ra3.4xlarge) is stuck at "Modifying (preparing for resize)" for many hours now. Is there something we can do to forcefully cancel the operation and/or reboot the cluster? Any pointers much appreciated -- thanks in advance!lg...

Hi Team, I have an existing redshift cluster, where I want to enable concurrency scaling. I had few queries related to the same : 1. My cluster having 2 on demand ra3.4xlarge nodes is running since March 2021. AWS docs mentions that the running redshift cluster accrues 1 hour of free cluster usage credit every 24 hours which never expire. Does it mean that, my cluster would already have 18 months * 30 credit usage hours already accrued, since concurrency scaling was never enabled for this cluster.? 2. When the does the concurrency scaling feature kicks in ? Is it only when the queries starts getting queued up ? Can we define some kind of threshold like cpu %utilization or memory % utilization, which would automatically start the concurrency scaling process ? 3. How much time does it take for cluster to complete the autoscaling process and start serving queries ? Thanks!lg...

Hello, I'm using running: ``` create model predict_xxxxx from (select col1, col2, col3 from my_table) target col3 function predict_xxx iam_role 'arn:aws:iam::xxxxxxx:role/RedshiftML' problem_type regression objective 'mse' settings ( s3_bucket 'redshiftml-xxxxxxx', s3_garbage_collect off, max_runtime 1800 ); ``` Which then generates input data files in CSV format in the S3 bucket I specified, but when I open up those files and look at them, all the columns in my `select` statement are present, but the column headers are mismatched with the data below them. I see `col1` data under the `col2` column and so on. I know the data is mixed up because the data types and numeric ranges are different for each column. I double-checked my table and the columns and data are matched correctly. Is Redshift/Sagemaker then using that mismatched data to train the model? I have tried with only two column and it still gets mixed up. I've tried using a table instead of a select expression and the problem persists. Any insight is appreciated. Thanks, - SVlg...

I've created a DMS task of CDC and Full Load, migrating data from postgres 14 to Redshift. According to the documentation, when using pglogical and creating postgres publication with 'publish_via_partition_root' parameter of my partitioned table, changes should be published to the parent table and to to child tables. However, the data is still migrated to the child tables in Redshift and not to the parent table. Am I missing something thats needs to be configured or is it just not possible in DMS?lg...