AWS Glue and REST API

2021-02-04T13:41:51

I am currently looking at ETL services to use to transform and push data to a data warehouse. I have come across AWS Glue and I am wondering if it is possible to extract data from external sources using REST API with AWS Glue. I have gone through the documentation and have not found out if it is possible. Does anyone know if this is possible and have any resources which I can refer to to point me in the right direction ?

Copyright License:
Author:「Maanoj Kumar」,Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.
Link to:https://stackoverflow.com/questions/66039934/aws-glue-and-rest-api

About “AWS Glue and REST API” questions

I am currently looking at ETL services to use to transform and push data to a data warehouse. I have come across AWS Glue and I am wondering if it is possible to extract data from external sources ...
I'm trying to create a workflow where AWS Glue ETL job will pull the JSON data from external REST API instead of S3 or any other AWS-internal sources. Is that even possible? Anyone does it? Please ...
I have setup aws glue crawlers and have already databases with tables populated to my Glue Data Catalog. I would like to access information on Data Catalog using Web API. For example I would like...
We are trying to evaluate AWS Gleu for ETL processing. In that line, there is a need to consume one of the external REST api from ETL job script. When job is run, Glue logs with Connection time out
I have a table which contains few schedules for various jobs. I want to process the records and create Triggers via AWS Glue API. http://docs.aws.amazon.com/glue/latest/dg/aws-glue-api.html The...
I am running an AWS Glue job written from scratch to read from database and save the result in s3. I would like to set an HTTP API call to send the status of the Glue job after completing the read ...
I am using AWS Glue and need to transform Boolean (True and False), columns within a Redshift datawarehouse schema to a "Yes"/"No" in another Redshift schema. At present, there does not appear to b...
AWS Glue Data Catalog is composed of different structures, e.g. Database, Table, Partition, Column etc. Haven't looked through every one of them, but it seems that Parameters fields (a map arra...
We are in a phase where we are migrating all of our spark job written in scala to aws glue. Current Flow: Apache Hive -> Spark(Processing/Transformation) -> Apache Hive -> BI Required Flow: AWS S3(
From aws documentation this one https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-catalog-tables.html, they mentioned this " Retention – Number (integer), not more than None. The retention ti...

Copyright License:Reproduced under the CC 4.0 BY-SA copyright license with link to original source & disclaimer.