1 Respuesta
- Más nuevo
- Más votos
- Más comentarios
0
The only standard way I know to do that is creating a symlink table (it has a manifest that points to the specific files, which you would need to update). But normally you would want to convert to columnar formats, standardize, transform, partitions or do other things before you create the table, take that chance to separate the files and generate good tables. I would save my users the pain of using CSV/JSON tables.
Contenido relevante
- OFICIAL DE AWSActualizada hace 3 años
- OFICIAL DE AWSActualizada hace 4 meses
- OFICIAL DE AWSActualizada hace 2 años
Could you please elaborate on this. I am not too familiar with the concept of a "symlink table".
And to do that, we want to use Athena, potentially via dbt. The idea was to define base raw tables, with partitions, then compact the data using Athena.
https://athena.guide/articles/stitching-tables-with-symlinktextinputformat