Differences

This shows you the differences between two versions of the page.

--- howtos:how_to_download_features_using_the_arcgis_rest_api [2024-07-21 09:58 am] – created hcho
+++ howtos:how_to_download_features_using_the_arcgis_rest_api [2024-09-24 07:35 am] (current) – hcho
@@ Line 1: / Line 1: @@
 ====== How to download features using the ArcGIS REST API ======
-This script is just an example of using the ArcGIS REST API, but if you know the URL, you can easily adapt it.
+This script requires [[https://jqlang.github.io/jq/|jq]].
-<code bash>
+<file bash download_arcgis_rest_features.sh>
+#!/bin/sh
+if [ $# -lt 2 ]; then
+	echo "Usage: download_arcgis_rest_features.sh REST_URL output"
+	exit
+fi
+# URL for an ArcGIS REST API
+rest_url=$1
+# output filename
+output=$2
+# prefix
+prefix=$(echo $output | sed 's/\..*//')
 # starting record offset
 offset=0
@@ Line 8: / Line 24: @@
 count=2000
-# URL for the NFHL ArcGIS REST API
+url="$rest_url/query"
-url="https://hazards.fema.gov/arcgis/rest/services/FIRMette/NFHLREST_FIRMette/MapServer/1/query?where=1%3D1&text=&objectIds=&time=&geometry=&geometryType=esriGeometryEnvelope&inSR=&spatialRel=esriSpatialRelIntersects&distance=&units=esriSRUnit_Foot&relationParam=&outFields=*&returnGeometry=true&returnTrueCurves=false&maxAllowableOffset=&geometryPrecision=&outSR=&havingClause=&returnIdsOnly=false&returnCountOnly=false&orderByFields=OBJECTID&groupByFieldsForStatistics=&outStatistics=&returnZ=false&returnM=false&gdbVersion=&historicMoment=&returnDistinctValues=false&resultOffset=---OFFSET---&resultRecordCount=$count&returnExtentOnly=false&datumTransformation=&parameterValues=&rangeValues=&quantizationParameters=&featureEncoding=esriDefault&f=geojson"
+# required parameters
+url="$url?f=geojson"
+url="$url&where=1%3D1"
+url="$url&outFields=*"
+url="$url&resultRecordCount=$count"
+url="$url&resultOffset=---OFFSET---"
+# optional parameters
+# &text=
+# &objectIds=
+# &time=
+# &geometry=
+# &geometryType=esriGeometryEnvelope
+# &inSR=
+# &spatialRel=esriSpatialRelIntersects
+# &distance=
+# &units=esriSRUnit_Foot
+# &relationParam=
+# &returnGeometry=true
+# &returnTrueCurves=false
+# &maxAllowableOffset=
+# &geometryPrecision=
+# &outSR=
+# &havingClause=
+# &returnIdsOnly=false
+# &returnCountOnly=false
+# &orderByFields=OBJECTID
+# &groupByFieldsForStatistics=
+# &outStatistics=
+# &returnZ=false
+# &returnM=false
+# &gdbVersion=
+# &historicMoment=
+# &returnDistinctValues=false
+# &returnExtentOnly=false
+# &datumTransformation=
+# &parameterValues=
+# &rangeValues=
+# &quantizationParameters=
+# &featureEncoding=esriDefault
 # infinite while loop; : mean true, so while : means while true, which means forever;
@@ Line 15: / Line 71: @@
 while :; do
         # generate a filename
-        fn=firm_panels_$offset.json
+        fn=${prefix}_$offset.geojson
         [ -f $fn ] || wget -O $fn "$(echo "$url" | sed "s/---OFFSET---/$offset/")"
 	# read wget error
@@ Line 33: / Line 89: @@
         offset=$(expr $offset + $count)
 done
+# merge geojson files
+jq 'reduce inputs.features as $s (.; .features += $s) | del(.properties)' *.geojson > $output
+</file>
+Here is an example:
+<code bash>
+./download_arcgis_rest_features.sh https://hazards.fema.gov/arcgis/rest/services/FIRMette/NFHLREST_FIRMette/MapServer/1 firm_panels
 </code>