Merge pull request #1 from rodekruis/convert_to_work_with_gfm

.env_template

@@ -0,0 +1,2 @@
+gfm_username=your-gfm-username
+gfm_password=your-gfm-password

.gitignore

@@ -1,3 +1,7 @@
+.env
+/output/*
+!/output/.gitkeep
+
 # Byte-compiled / optimized / DLL files
 __pycache__/
 *.py[cod]

.pre-commit-config.yaml

@@ -1,39 +0,0 @@
-default_language_version:
-    python: python3.9
-
-repos:
-  - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v4.4.0
-    hooks:
-      - id: trailing-whitespace
-      - id: end-of-file-fixer
-      - id: check-ast
-      - id: check-toml
-      - id: check-yaml
-  - repo: https://github.com/psf/black
-    rev: 23.1.0
-    hooks:
-      - id: black
-  - repo: https://github.com/PyCQA/isort
-    rev: 5.12.0
-    hooks:
-      - id: isort
-  - repo: https://github.com/pycqa/flake8
-    rev: 6.0.0
-    hooks:
-      - id: flake8
-        additional_dependencies:
-        #   - flake8-bugbear==21.9.2
-        #   - flake8-comprehensions==3.6.1
-          # - flake8-deprecated==1.3
-          - flake8-docstrings==1.6.0
-          # - flake8-keyword-arguments==0.1.0
-  - repo: https://github.com/pre-commit/mirrors-mypy
-    rev: v0.910
-    hooks:
-      - id: mypy
-        additional_dependencies:
-          - types-PyYAML==5.4.10
-          - types-setuptools==57.4.0
-          - types-requests==2.25.9
-        exclude: tests

README.md

@@ -1,63 +1,15 @@
 # Flood Mapping Tool
 
-[![Open in Streamlit](https://static.streamlit.io/badges/streamlit_badge_black_white.svg)](https://mapaction-flood-map.streamlit.app/)
-[![license](https://img.shields.io/github/license/OCHA-DAP/pa-aa-toolbox.svg)](https://github.com/mapaction/flood-mapping-tool/blob/main/LICENSE)
-[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
-[![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat&labelColor=ef8336)](https://pycqa.github.io/isort/)
+## Installation & Running
+### GFM account
+To run the app you will need a (free) GFM account. To get one register at https://portal.gfm.eodc.eu/login . Once you have an account create a file `.env` with the content of `.env_template` where you will need to fill out your GFM username and password. The `.env` file will be gitignored.
 
-This repository contains a Streamlit app that allows to estimate flood extent using Sentinel-1 synthetic-aperture radar <a href='https://sentinel.esa.int/web/sentinel/user-guidessentinel-1-sar'>SAR</a> data.
+### Python
+Project is using python 3.12. Install requirements from `pyproject.toml` in your preferred way. We suggest using `uv`, see [here](https://docs.astral.sh/uv/getting-started/installation/) for installation instructions. Once installed your can run `uv sync` to create a `.venv` folder. Activate the `.venv` and then run the line below to run the app:
 
-The methodology is based on a <a href='https://un-spider.org/advisory-support/recommended-practices/recommended-\practice-google-earth-engine-flood-mapping'> recommended practice </a> published by the United Nations Platform for Space-based Information for Disaster Management and Emergency Response (UN-SPIDER) and it uses several satellite imagery datasets to produce the final output. The datasets are retrieved from <a href='https://earthengine.google.com/'>Google Earth Engine</a> which is a powerful web-platform for cloud-based processing of remote sensing data on large scales. More information on the methodology is given in the <i>Description</i> page in the Streamlit app.
-
-This analysis provides a comprehensive overview of a flooding event, across different areas of interest, from settlements to countries. However, as mentioned in the UN-SPIDER website, the methodology is meant for broad information provision in a global context, and contains inherent uncertainties. Therefore, it is important that the tool is not used as the only source of information for rescue response planning.
-
-## Usage
-
-#### Requirements
-
-The Python version currently used is 3.10. Please install all packages from
-``requirements.txt``:
-
-```shell
-pip install -r requirements.txt
 ```
-
-#### Google Earth Engine authentication
-
-[Sign up](https://signup.earthengine.google.com/) for a Google Earth Engine account, if you don't already have one. Open a terminal window, type `python` and then paste the following code:
-
-```python
-import ee
-ee.Authenticate()
-```
-
-Log in to your Google account to obtain the authorization code and paste it back into the terminal. Once you press "Enter", an authorization token will be saved to your computer under the following file path (depending on your operating system):
-
-- Windows: `C:\\Users\\USERNAME\\.config\\earthengine\\credentials`
-- Linux: `/home/USERNAME/.config/earthengine/credentials`
-- MacOS: `/Users/USERNAME/.config/earthengine/credentials`
-
-The credentials will be used when initialising Google Earth Engine in the app.
-
-#### Run the app
-
-Finally, open a terminal and run
-
-```shell
 streamlit run app/Home.py
 ```
 
-A new browser window will open and you can start using the tool.
-
-## Contributing
-
-#### Pre-commit
-
-All code is formatted according to
-[black](https://github.com/psf/black) and [flake8](https://flake8.pycqa.org/en/latest) guidelines. The repo is set-up to use [pre-commit](https://github.com/pre-commit/pre-commit). Please run ``pre-commit install`` the first time you are editing. Thereafter all commits will be checked against black and flake8 guidelines.
-
-To check if your changes pass pre-commit without committing, run:
-
-```shell
-pre-commit run --all-files
-```
+## Project
+TODO: Add more complete documentation. 

app/Home.py

@@ -1,21 +1,21 @@
 """Home page for Streamlit app."""
+
 import streamlit as st
 from src.config_parameters import params
+
+# Page configuration
+st.set_page_config(layout="wide", page_title=params["browser_title"])
+
 from src.utils import (
     add_about,
-    add_logo,
     set_home_page_style,
     toggle_menu_button,
 )
 
-# Page configuration
-st.set_page_config(layout="wide", page_title=params["browser_title"])
-
 # If app is deployed hide menu button
 toggle_menu_button()
 
 # Create sidebar
-add_logo("app/img/MA-logo.png")
 add_about()
 
 # Set page style
@@ -26,110 +26,8 @@ st.markdown("# Home")
 
 # First section
 st.markdown("## Introduction")
-st.markdown(
-    """
-    This tool allows to estimate flood extent using Sentinel-1
-    synthetic-aperture radar
-    <a href='%s'>SAR</a> data.<br><br>
-    The methodology is based on a <a href=
-    '%s'>recommended practice</a>
-    published by the United Nations Platform for Space-based Information for
-    Disaster Management and Emergency Response (UN-SPIDER) and it uses several
-    satellite imagery datasets to produce the final output. The datasets are
-    retrieved from <a href='%s'>Google Earth
-    Engine</a> which is a powerful web-platform for cloud-based processing of
-    remote sensing data on large scales. More information on the methodology is
-    given in the Description.<br><br>
-    This analysis provides a comprehensive overview of a flooding event, across
-    different areas of interest, from settlements to countries. However, as
-    mentioned in the UN-SPIDER website, the methodology is meant for broad
-    information provision in a global context, and contains inherent
-    uncertainties. Therefore, it is important that the tool is not used as the
-    only source of information for rescue response planning.
-    """
-    % (
-        params["url_sentinel_esa"],
-        params["url_unspider_tutorial"],
-        params["url_gee"],
-    ),
-    unsafe_allow_html=True,
-)
+st.markdown("TODO: new introduction")
 
 # Second section
 st.markdown("## How to use the tool")
-st.markdown(
-    """
-    <ul>
-        <li><p>
-            In the sidebar, choose <i>Flood extent analysis</i> to start the
-            analysis.
-        </p>
-        <li><p>
-            In the left panel, use the drawing tool to select an area of
-            interest on the map. You can delete your selection by clicking on
-            the bin icon. While the flood mapping is generated regardless of
-            the size of the selected region, you will be able to save raster
-            and vector flooding extent only if the side of the rectangular
-            selection does not exceed 100 km.
-        </p>
-        <li><p>
-            In the right panel click on the title <i>Choose Image Dates</i>
-            in order to expand the section. Here you need to select four dates.
-            The first two identify a range of dates based on which the
-            reference imagery (before the flooding event) is defined. You can
-            select even years worth of data (the reference imagery is
-            calculated as the median between the range of observations), but
-            make sure you take into account wet and dry seasons if only taking
-            a few months. The last two refer to a period of time which comes
-            after the flooding event. By setting periods, not single dates, you
-            allow the selection of enough tiles to cover the area of interest.
-            Sentinel-1 imagery is acquired minimum every 12 days for each point
-            on the globe (see Figure 2 in the documentation).
-        </p>
-        <li>
-            <p>
-                By clicking on <i>Choose parameters</i>, you will be able to
-                set two variables:
-            </p>
-            <ul>
-                <li><p>
-                    The <i>threshold</i> is the value against which the
-                    difference the two satellite images - before and after the
-                    flooding event - is tested. Lower thresholds result in a
-                    greater area considered "flooded". It is recommended to set
-                    the value to 1.25, which was selected through trial and
-                    error. You may want to adjust the value in case of high
-                    rates of false positive or negative values, especially in
-                    case other sources of information are available and it is
-                    possible to compare flood extent estimations between
-                    sources.
-                </p>
-                <li><p>
-                    The <i>pass direction</i> has to do with the way the
-                    satellite travels around the Earth. Depending on your area
-                    of interest and time period, you may find more imagery
-                    available for either the <i>Ascending</i> or the
-                    <i>Descending</i> pass directions (see Figure 2 in the
-                    Documentation). It is recommended to leave the parameter
-                    unchanged for a first estimation and change its value in
-                    case partial or no imagery is produced.
-                </p>
-            </ul>
-        <li><p>
-            Once the parameters are set, you can finally click on <i>Compute
-            flood extent</i> to run the calculations. A map will appear
-            underneath, with a layer containing the flooded area within the
-            area of interest.
-        </p>
-        <li><p>
-            If you wish to export the layer to file, you can click on <i>Export
-            to file</i> and download the raster and/or vector data.
-        </p>
-    </ul>
-    <p>
-        In case you get errors, follow the intructions. If you have doubts,
-        feel free to contact the Data Science team.
-    </p>
-    """,
-    unsafe_allow_html=True,
-)
+st.markdown("TODO: new how to use the tool")

app/pages/0_AOIs.py

@@ -0,0 +1,169 @@
+import json
+import os
+from pathlib import Path
+
+import folium
+import requests
+import streamlit as st
+from folium.plugins import Draw
+from src.config_parameters import params
+from src.gfm import download_gfm_geojson, get_existing_flood_geojson, sync_cached_data
+from src.utils import (
+    add_about,
+    set_tool_page_style,
+    toggle_menu_button,
+)
+from streamlit_folium import st_folium
+
+# Page configuration
+st.set_page_config(layout="wide", page_title=params["browser_title"])
+
+# If app is deployed hide menu button
+toggle_menu_button()
+
+# Create sidebar
+add_about()
+
+# Page title
+st.markdown("# Flood extent analysis")
+
+# Set page style
+set_tool_page_style()
+
+# Create two rows: top and bottom panel
+row1 = st.container()
+save_area = False
+
+with row1:
+    action_type = st.radio(
+        label="Action Type",
+        options=["See Areas", "Create New Area", "Delete Area"],
+        label_visibility="hidden",
+    )
+
+    # call to render Folium map in Streamlit
+    folium_map = folium.Map([39, 0], zoom_start=8)
+    feat_group_selected_area = folium.FeatureGroup(name="selected_area")
+
+    if action_type == "See Areas":
+        with open("./bboxes/bboxes.json", "r") as f:
+            bboxes = json.load(f)
+        for area_name in bboxes.keys():
+            bbox = bboxes[area_name]["bounding_box"]
+            feat_group_selected_area.add_child(folium.GeoJson(bbox))
+
+        folium_map.fit_bounds(feat_group_selected_area.get_bounds())
+
+    elif action_type == "Create New Area":
+        Draw(
+            export=False,
+            draw_options={
+                "circle": False,
+                "polyline": False,
+                "polygon": False,
+                "rectangle": True,
+                "marker": False,
+                "circlemarker": False,
+            },
+        ).add_to(folium_map)
+
+        new_area_name = st.text_input("Area name")
+        save_area = st.button("Save Area")
+
+    elif action_type == "Delete Area":
+        # Load existing bboxes
+        with open("./bboxes/bboxes.json", "r") as f:
+            bboxes = json.load(f)
+            existing_areas = bboxes.keys()
+
+        area_to_delete = st.selectbox("Choose area to delete", options=existing_areas)
+        bbox = bboxes[area_to_delete]["bounding_box"]
+        feat_group_selected_area.add_child(folium.GeoJson(bbox))
+        folium_map.fit_bounds(feat_group_selected_area.get_bounds())
+
+        delete_area = st.button("Delete")
+
+        if delete_area:
+            with open("./bboxes/bboxes.json", "r") as f:
+                bboxes = json.load(f)
+            bboxes.pop(area_to_delete, None)
+            with open("./bboxes/bboxes.json", "w") as f:
+                json.dump(bboxes, f)
+            st.toast("Area successfully deleted")
+
+    with open("./bboxes/bboxes.json", "r") as f:
+        bboxes = json.load(f)
+
+    # geojson_catania = get_existing_flood_geojson("Catania")
+    # print(geojson_catania)
+    # geojson_selected_area = folium.GeoJson(geojson_catania)
+
+    # feat_group_selected_area.add_child(geojson_selected_area)
+    m = st_folium(
+        folium_map,
+        width=800,
+        height=450,
+        feature_group_to_add=feat_group_selected_area,
+    )
+
+    if save_area:
+        check_drawing = m["all_drawings"] != [] and m["all_drawings"] is not None
+        if not check_drawing:
+            st.error("Please create a region using the rectangle tool on the map.")
+        elif new_area_name == "":
+            st.error("Please provide a name for the new area")
+        else:
+            # Get the drawn area
+            selected_area_geojson = m["all_drawings"][-1]
+
+            print("starting to post new area name to gfm api")
+            coordinates = selected_area_geojson["geometry"]["coordinates"]
+
+            username = os.environ["gfm_username"]
+            password = os.environ["gfm_password"]
+            base_url = "https://api.gfm.eodc.eu/v1"
+
+            # Get token, setup header
+            token_url = f"{base_url}/auth/login"
+
+            payload = {"email": username, "password": password}
+
+            response = requests.post(token_url, json=payload)
+            user_id = response.json()["client_id"]
+            access_token = response.json()["access_token"]
+            header = {"Authorization": f"bearer {access_token}"}
+
+            print("authenticated to API")
+            # Create area of impact
+            create_aoi_url = f"{base_url}/aoi/create"
+
+            payload = {
+                "aoi_name": new_area_name,
+                "description": new_area_name,
+                "user_id": user_id,
+                "geoJSON": {"type": "Polygon", "coordinates": coordinates},
+            }
+
+            r = requests.post(url=create_aoi_url, json=payload, headers=header)
+            print(r.json())
+            print("Posted new AOI")
+
+            print("Writing new area to bbox json")
+            # Load existing bboxes
+            with open("./bboxes/bboxes.json", "r") as f:
+                bboxes = json.load(f)
+
+            # If the    area doesn't exist, create it
+            if new_area_name not in bboxes:
+                bboxes[new_area_name] = {}
+
+            # Save the new bounding box under the date range key
+            bboxes[new_area_name] = {
+                "bounding_box": selected_area_geojson,
+                "date_ranges": [],  # Will be populated when files are downloaded
+            }
+            # Write the updated data back to file
+            with open("./bboxes/bboxes.json", "w") as f:
+                json.dump(bboxes, f, indent=4)
+
+            st.toast("Area successfully created")

app/pages/1_🌍_Flood_extent_analysis.py

@@ -1,22 +1,16 @@
-"""Flood extent analysis page for Streamlit app."""
-import datetime as dt
+import json
+from pathlib import Path
 
-import ee
 import folium
-import geemap.foliumap as geemap
-import requests
 import streamlit as st
-import streamlit_ext as ste
-from folium.plugins import Draw, Geocoder, MiniMap
+from folium.plugins import Draw
 from src.config_parameters import params
+from src.gfm import download_gfm_geojson, get_existing_flood_geojson, sync_cached_data
 from src.utils import (
     add_about,
-    add_logo,
     set_tool_page_style,
     toggle_menu_button,
 )
-from src.utils_ee import ee_initialize
-from src.utils_flood_analysis import derive_flood_extents
 from streamlit_folium import st_folium
 
 # Page configuration
@@ -26,7 +20,6 @@ st.set_page_config(layout="wide", page_title=params["browser_title"])
 toggle_menu_button()
 
 # Create sidebar
-add_logo("app/img/MA-logo.png")
 add_about()
 
 # Page title
@@ -35,268 +28,103 @@ st.markdown("# Flood extent analysis")
 # Set page style
 set_tool_page_style()
 
-# Initialise Google Earth Engine
-ee_initialize(force_use_service_account=True)
-
-
-# Output_created is useful to decide whether the bottom panel with the
-# output map should be visualised or not
-if "output_created" not in st.session_state:
-    st.session_state.output_created = False
-
-
-# Function to be used to hide bottom panel (when setting parameters for a
-# new analysis)
-def callback():
-    """Set output created to zero: reset tool."""
-    st.session_state.output_created = False
-
+# Sync cached data
+# ! WARNING: will erase your output folder
+# # # sync_cached_data()
 
 # Create two rows: top and bottom panel
 row1 = st.container()
-row2 = st.container()
 # Crate two columns in the top panel: input map and paramters
 col1, col2 = row1.columns([2, 1])
+feat_group_selected_area = folium.FeatureGroup(name="selected_area")
 with col1:
+    with open("./bboxes/bboxes.json", "r") as f:
+        bboxes = json.load(f)
+    selected_area = st.selectbox("Select saved area", options=bboxes.keys())
+
+    # retrieve and select available dates
+    if selected_area:
+        area_folder = Path(f"./output/{selected_area}")
+        if area_folder.exists():
+            available_product_times = [
+                str(f.name) for f in area_folder.iterdir() if f.is_dir()
+            ]
+
+            available_product_times = [
+                prod_time.replace("_", ":") for prod_time in available_product_times
+            ]
+            selected_product_time = st.selectbox(
+                "Select available date range", options=available_product_times
+            )
+        else:
+            selected_product_time = None
+
+        # display the bounding box
+        bounding_box = bboxes[selected_area]["bounding_box"]
+        geojson_selected_area = folium.GeoJson(bounding_box)
+        feat_group_selected_area.add_child(geojson_selected_area)
+
+        # geojson_selected_area = folium.GeoJson(bboxes[selected_area])
+        # feat_group_selected_area.add_child(geojson_selected_area)
+        if selected_product_time:
+            geojson_flood_area = get_existing_flood_geojson(
+                selected_area, selected_product_time
+            )
+            feat_group_selected_area.add_child(geojson_flood_area)
+
     # Add collapsable container for input map
     with st.expander("Input map", expanded=True):
         # Create folium map
-        Map = folium.Map(
-            location=[52.205276, 0.119167],
-            zoom_start=3,
-            control_scale=True,
-            # crs='EPSG4326'
+        # call to render Folium map in Streamlit
+        folium_map = folium.Map([39, 0], zoom_start=8)
+        # check if the FeatureGroup has any children (i.e., layers added)
+        if len(feat_group_selected_area._children) > 0:
+            # if there is data, fit the map to the bounds
+            folium_map.fit_bounds(feat_group_selected_area.get_bounds())
+        else:
+            # if there is no data, set a default view
+            # this is necessary to start up the page
+            folium_map = folium.Map(location=[39, 0], zoom_start=8)
+
+        m = st_folium(
+            folium_map,
+            width=800,
+            height=450,
+            feature_group_to_add=feat_group_selected_area,
         )
-        # Add drawing tools to map
-        Draw(
-            export=False,
-            draw_options={
-                "circle": False,
-                "polyline": False,
-                "polygon": True,
-                "circle": False,
-                "marker": False,
-                "circlemarker": False,
-            },
-        ).add_to(Map)
-        # Add search bar with geocoder to map
-        Geocoder(add_marker=False).add_to(Map)
-        # Add minimap to map
-        MiniMap().add_to(Map)
-        # Export map to Streamlit
-        output = st_folium(Map, width=800, height=600)
 with col2:
     # Add collapsable container for image dates
-    with st.expander("Choose Image Dates"):
-        # Callback is added, so that, every time a parameters is changed,
-        # the bottom panel containing the output map is hidden
-        before_start = st.date_input(
-            "Start date for reference imagery",
-            value=dt.date(year=2022, month=7, day=1),
-            help="It needs to be prior to the flooding event",
-            on_change=callback,
-        )
-        before_end = st.date_input(
-            "End date for reference imagery",
-            value=dt.date(year=2022, month=7, day=30),
-            help=(
-                "It needs to be prior to the flooding event, at least 15 "
-                "days subsequent to the date selected above"
-            ),
-            on_change=callback,
-        )
-        after_start = st.date_input(
-            "Start date for flooding imagery",
-            value=dt.date(year=2022, month=9, day=1),
-            help="It needs to be subsequent to the flooding event",
-            on_change=callback,
-        )
-        after_end = st.date_input(
-            "End date for flooding imagery",
-            value=dt.date(year=2022, month=9, day=16),
-            help=(
-                "It needs to be subsequent to the flooding event and at "
-                "least 10 days to the date selected above"
-            ),
-            on_change=callback,
-        )
+    with st.expander("Choose Dates", expanded=True):
+        start_date = st.date_input("Start date")
+        end_date = st.date_input("End date")
     # Add collapsable container for parameters
     with st.expander("Choose Parameters"):
         # Add slider for threshold
-        add_slider = st.slider(
-            label="Select a threshold",
-            min_value=0.0,
-            max_value=5.0,
-            value=1.25,
-            step=0.25,
-            help="Higher values might reduce overall noise",
-            on_change=callback,
-        )
-        # Add radio buttons for pass direction
-        pass_direction = st.radio(
-            "Set pass direction",
-            ["Ascending", "Descending"],
-            on_change=callback,
-        )
-    # Button for computation
-    submitted = st.button("Compute flood extent")
-    # Introduce date validation
-    check_dates = before_start < before_end <= after_start < after_end
-    # Introduce drawing validation (a polygon needs to exist)
-    check_drawing = (
-        output["all_drawings"] != [] and output["all_drawings"] is not None
-    )
-# What happens when button is clicked on?
+        st.text("Add relevant (API) parameters here")
+
+    submitted = st.button("Update flood extent")
+
+
+# If the button is clicked do the following
 if submitted:
     with col2:
+        # Some basic validation on dates and that there's an area if relevant
+        get_gfm = True
+        check_dates = start_date <= end_date
+
         # Output error if dates are not valid
         if not check_dates:
             st.error("Make sure that the dates were inserted correctly")
-        # Output error if no polygons were drawn
-        elif not check_drawing:
-            st.error("No region selected.")
-        else:
-            # Add output for computation
-            with st.spinner("Computing... Please wait..."):
-                # Extract coordinates from drawn polygon
-                coords = output["all_drawings"][-1]["geometry"]["coordinates"][
-                    0
-                ]
-                # Create geometry from coordinates
-                ee_geom_region = ee.Geometry.Polygon(coords)
-                # Crate flood raster and vector
-                (
-                    detected_flood_vector,
-                    detected_flood_raster,
-                    _,
-                    _,
-                ) = derive_flood_extents(
-                    aoi=ee_geom_region,
-                    before_start_date=str(before_start),
-                    before_end_date=str(before_end),
-                    after_start_date=str(after_start),
-                    after_end_date=str(after_end),
-                    difference_threshold=add_slider,
-                    polarization="VH",
-                    pass_direction=pass_direction,
-                    export=False,
-                )
-                # Create output map
-                Map2 = geemap.Map(
-                    # basemap="HYBRID",
-                    plugin_Draw=False,
-                    Draw_export=False,
-                    locate_control=False,
-                    plugin_LatLngPopup=False,
+            get_gfm = False
+
+        # Only if checks pass go and get the GFM data
+        if get_gfm:
+            # Show loader because it will take a while
+            with st.spinner("Getting GFM files... Please wait..."):
+                # download_gfm_geojson(area_name)
+                download_gfm_geojson(
+                    selected_area, from_date=start_date, to_date=end_date
                 )
-                try:
-                    # Add flood vector layer to map
-                    Map2.add_layer(
-                        ee_object=detected_flood_raster,
-                        name="Flood extent raster",
-                    )
-                    Map2.add_layer(
-                        ee_object=detected_flood_vector,
-                        name="Flood extent vector",
-                    )
-                    # Center map on flood raster
-                    Map2.centerObject(detected_flood_raster)
-                except ee.EEException:
-                    # If error contains the sentence below, it means that
-                    # an image could not be properly generated
-                    st.error(
-                        """
-                        No satellite image found for the selected
-                        dates.\n\n
-                        Try changing the pass direction.\n\n
-                        If this does not work, choose different
-                        dates: it is likely that the satellite did not
-                        cover the area of interest in the range of
-                        dates specified (either before or after the
-                        flooding event).
-                        """
-                    )
-                else:
-                    # If computation was succesfull, save outputs for
-                    # output map
-                    st.success("Computation complete")
-                    st.session_state.output_created = True
-                    st.session_state.Map2 = Map2
-                    st.session_state.detected_flood_raster = (
-                        detected_flood_raster
-                    )
-                    st.session_state.detected_flood_vector = (
-                        detected_flood_vector
-                    )
-                    st.session_state.ee_geom_region = ee_geom_region
-# If computation was successful, create output map in bottom panel
-if st.session_state.output_created:
-    with row2:
-        # Add collapsable container for output map
-        with st.expander("Output map", expanded=True):
-            # Export Map2 to streamlit
-            st.session_state.Map2.to_streamlit()
-            # Create button to export to file
-            submitted2 = st.button("Export to file")
-            # What happens if button is clicked on?
-            if submitted2:
-                # Add output for computation
-                with st.spinner("Computing... Please wait..."):
-                    try:
-                        # Get download url for raster data
-                        raster = st.session_state.detected_flood_raster
-                        url_r = raster.getDownloadUrl(
-                            {
-                                "region": st.session_state.ee_geom_region,
-                                "scale": 30,
-                                "format": "GEO_TIFF",
-                            }
-                        )
-                    except Exception:
-                        st.error(
-                            """
-                            The image size is too big for the image to
-                            be exported to file. Select a smaller area
-                            of interest (side <~ 150km) and repeat the
-                            analysis.
-                            """
-                        )
-                    else:
-                        response_r = requests.get(url_r)
-                        # Get download url for raster data
-                        vector = st.session_state.detected_flood_vector
-                        url_v = vector.getDownloadUrl("GEOJSON")
-                        response_v = requests.get(url_v)
-                        filename = "flood_extent"
-                        timestamp = dt.datetime.now().strftime(
-                            "%Y-%m-%d_%H-%M"
-                        )
-                        with row2:
-                            # Create download buttons for raster and vector
-                            # data
-                            with open("flood_extent.tif", "wb"):
-                                ste.download_button(
-                                    label="Download Raster Extent",
-                                    data=response_r.content,
-                                    file_name=(
-                                        f"{filename}"
-                                        "_raster_"
-                                        f"{timestamp}"
-                                        ".tif"
-                                    ),
-                                    mime="image/tif",
-                                )
-                            with open("flood_extent.geojson", "wb"):
-                                ste.download_button(
-                                    label="Download Vector Extent",
-                                    data=response_v.content,
-                                    file_name=(
-                                        f"{filename}"
-                                        "_vector_"
-                                        f"{timestamp}"
-                                        ".geojson"
-                                    ),
-                                    mime="text/json",
-                                )
-                        # Output for computation complete
-                        st.success("Computation complete")
+
+            # Display that getting the files is finished
+            st.markdown("Getting GFM files finished")

app/pages/2_📖_Documentation.py

@@ -1,10 +1,9 @@
 """Documentation page for Streamlit app."""
+
 import streamlit as st
-from PIL import Image
 from src.config_parameters import params
 from src.utils import (
     add_about,
-    add_logo,
     set_doc_page_style,
     toggle_menu_button,
 )
@@ -16,7 +15,6 @@ st.set_page_config(layout="wide", page_title=params["browser_title"])
 toggle_menu_button()
 
 # Create sidebar
-add_logo("app/img/MA-logo.png")
 add_about()
 
 # Set page style
@@ -28,63 +26,9 @@ st.markdown("# Documentation")
 # First section
 st.markdown("## Methodology")
 st.markdown(
-    """
-    The methodology is based on the workflow depicted in Figure 1. In
-    addition to Sentinel-1 synthetic-aperture radar <a href='%s'>SAR</a> data,
-    two other datasets are used through <a href='%s'>Google Earth Engine</a>:
-    <ul>
-        <li><p>
-            The <i>WWF HydroSHEDS Void-Filled DEM, 3 Arc-Seconds</i>
-            <a href='%s'>dataset</a> is based on elevation data
-            obtained in 2000 by NASA's Shuttle Radar Topography Mission (SRTM),
-            and it is used to mask out areas with more than 5 percent slope
-            (see following section on limitations).
-        </p>
-        <li><p>
-            The <i>JRC Global Surface Water Mapping Layers, v1.4</i>
-            <a href='%s'>dataset</a> contains maps of the
-            location and temporal distribution of surface water from 1984 to
-            2021, and it is used to mask areas with perennial water bodies,
-            such as rivers or lakes.
-        </p>
-    </ul>
-    """
-    % (
-        params["url_sentinel_dataset"],
-        params["url_gee"],
-        params["url_elevation_dataset"],
-        params["url_surface_water_dataset"],
-    ),
-    unsafe_allow_html=True,
+    "TODO: new documentation, only kept in Sentinel 1 section unchanged from the Mapaction tool"
 )
 
-# Add image workflow
-img = Image.open("app/img/workflow.png")
-col1, mid, col2, last = st.columns([5, 3, 10, 10])
-with col1:
-    st.image(img, width=350)
-with col2:
-    # Trick to add caption at the bottom of the column, as Streamlit has not
-    # developed a functionality to allign text to bottom
-    space_before_caption = "<br>" * 27
-    st.markdown(
-        space_before_caption,
-        unsafe_allow_html=True,
-    )
-    st.markdown(
-        """
-        <p style="font-size:%s;">
-            Figure 1. Workflow of the flood mapping methodology (<a href=
-            '%s'>source</a>).
-        </p>
-        """
-        % (
-            params["docs_caption_fontsize"],
-            params["url_unspider_tutorial_detail"],
-        ),
-        unsafe_allow_html=True,
-    )
-
 
 # Second section
 st.markdown("## Radar imagery for flood detection")
@@ -140,40 +84,3 @@ st.markdown(
     % (params["docs_caption_fontsize"], params["url_sentinel_img_location"]),
     unsafe_allow_html=True,
 )
-
-# Third section
-st.markdown("## Key limitations")
-st.markdown(
-    """
-    Radar imagery is great for detecting floods, as it is good at picking up
-    water and it is not affected by the time of the day or clouds (at this
-    wavelength). But it has its limits, and performs actually quite bad if
-    having to detect water in mountainous regions, especially if with narrow
-    valleys, and in urban areas (urban canyons). The reasons are mainly around
-    the viewing angles, which can cause image distortions. This method may also
-    result in false positives for other land cover changes with smooth
-    surfaces, such as roads and sand. Rough surface texture caused by wind or
-    rainfall may also make it challenging for the radar imagery to identify
-    water bodies.
-    """,
-    unsafe_allow_html=True,
-)
-
-
-# Last section
-st.markdown("## Useful links")
-st.markdown(
-    """
-    <a href='%s'>UN-SPIDER recommended practice</a><br>
-    <a href='%s'>Sentinel-1 satellite imagery user guide</a><br>
-    Relevant scientific publications:
-    <a href='%s'>1</a>, <a href='%s'>2</a><br>
-    """
-    % (
-        params["url_unspider_tutorial"],
-        params["url_sentinel_esa"],
-        params["url_publication_1"],
-        params["url_publication_2"],
-    ),
-    unsafe_allow_html=True,
-)

app/src/config_parameters.py

@@ -1,65 +1,26 @@
 """Configuration file."""
+
 params = {
     # Title browser tab
-    "browser_title": "Flood mapping tool - MapAction",
+    "browser_title": "Flood mapping tool - 510",
     # Data scientists involved
     "data_scientists": {
-        "Piet": "pgerrits@mapaction.org",
-        "Daniele": "[email protected]",
-        "Cate": "[email protected]",
+        "Daniele": "dcastellana@redcross.nl",
     },
     # Urls
-    "url_data_science_wiki": (
-        "https://mapaction.atlassian.net/wiki/spaces/GAFO/overview"
-    ),
-    "url_gee": "https://earthengine.google.com/",
-    "url_project_wiki": (
-        "https://mapaction.atlassian.net/wiki/spaces/GAFO/pages/15920922751/"
-        "Rapid+flood+mapping+from+satellite+imagery"
-    ),
     "url_github_repo": "https://github.com/mapaction/flood-extent-tool",
-    "url_sentinel_esa": (
-        "https://sentinel.esa.int/web/sentinel/user-guides/sentinel-1-sar"
-    ),
     "url_sentinel_dataset": (
-        "https://developers.google.com/earth-engine/datasets/catalog/"
-        "COPERNICUS_S1_GRD"
+        "https://developers.google.com/earth-engine/datasets/catalog/COPERNICUS_S1_GRD"
     ),
     "url_sentinel_img": (
         "https://sentinel.esa.int/documents/247904/4748961/Sentinel-1-Repeat-"
         "Coverage-Frequency-Geometry-2021.jpg"
     ),
     "url_sentinel_img_location": (
-        "https://sentinel.esa.int/web/sentinel/missions/sentinel-1/"
-        "observation-scenario"
-    ),
-    "url_unspider_tutorial": (
-        "https://un-spider.org/advisory-support/recommended-practices/"
-        "recommended-practice-google-earth-engine-flood-mapping"
-    ),
-    "url_unspider_tutorial_detail": (
-        "https://un-spider.org/advisory-support/recommended-practices/"
-        "recommended-practice-google-earth-engine-flood-mapping/in-detail"
-    ),
-    "url_elevation_dataset": (
-        "https://developers.google.com/earth-engine/datasets/catalog/"
-        "WWF_HydroSHEDS_03VFDEM"
-    ),
-    "url_surface_water_dataset": (
-        "https://developers.google.com/earth-engine/datasets/catalog/"
-        "JRC_GSW1_4_GlobalSurfaceWater"
-    ),
-    "url_publication_1": (
-        "https://onlinelibrary.wiley.com/doi/full/10.1111/jfr3.12303"
-    ),
-    "url_publication_2": (
-        "https://www.sciencedirect.com/science/article/abs/pii/"
-        "S0924271620301702"
+        "https://sentinel.esa.int/web/sentinel/missions/sentinel-1/observation-scenario"
     ),
     # Layout and styles
     ## Sidebar
-    "MA_logo_width": "60%",
-    "MA_logo_background_position": "35% 10%",
     "sidebar_header": "Flood Mapping Tool",
     "sidebar_header_fontsize": "30px",
     "sidebar_header_fontweight": "bold",

app/src/gfm.py

@@ -0,0 +1,144 @@
+import io
+import json
+import os
+import zipfile
+from pathlib import Path
+
+import folium
+import requests
+from dotenv import load_dotenv
+
+load_dotenv()
+
+
+class FloodGeoJsonError(Exception):
+    """Custom exception for errors in fetching flood GeoJSON files."""
+
+    pass
+
+
+def sync_cached_data(bboxes_path="./bboxes/bboxes.json", output_dir="./output"):
+    """
+    Ensures that all areas in bboxes.json have a corresponding folder in ./output/.
+    Removes any area entry from bboxes.json that does not have an output folder.
+    """
+    try:
+        # Load existing bounding boxes
+        with open(bboxes_path, "r") as f:
+            bboxes = json.load(f)
+
+        # Get a set of existing output folders
+        existing_folders = {
+            folder.name for folder in Path(output_dir).iterdir() if folder.is_dir()
+        }
+
+        # Remove entries from bboxes.json if the folder does not exist
+        updated_bboxes = {
+            area: data for area, data in bboxes.items() if area in existing_folders
+        }
+
+        # If changes were made, overwrite bboxes.json
+        if len(updated_bboxes) != len(bboxes):
+            with open(bboxes_path, "w") as f:
+                json.dump(updated_bboxes, f, indent=4)
+            print(f"Updated {bboxes_path}: Removed missing areas.")
+        else:
+            print("All areas have matching folders.")
+
+    except FileNotFoundError:
+        print(f"Error: {bboxes_path} not found.")
+    except json.JSONDecodeError:
+        print(f"Error: {bboxes_path} is not a valid JSON file.")
+
+
+def download_gfm_geojson(
+    area_name, from_date=None, to_date=None, output_file_path=None
+):
+    """
+    Should provide an existing area name or a new area name with new_coordinates
+    """
+    username = os.environ["gfm_username"]
+    password = os.environ["gfm_password"]
+    base_url = "https://api.gfm.eodc.eu/v1"
+
+    # Get token, setup header
+    token_url = f"{base_url}/auth/login"
+
+    payload = {"email": username, "password": password}
+
+    response = requests.post(token_url, json=payload)
+    user_id = response.json()["client_id"]
+    access_token = response.json()["access_token"]
+    header = {"Authorization": f"bearer {access_token}"}
+    print("logged in")
+
+    # Get Area of Impact
+    aoi_url = f"{base_url}/aoi/user/{user_id}"
+    response = requests.get(aoi_url, headers=header)
+
+    # TODO: now only getting the first AOI, should extend to getting the whole list and unioning the geojsons
+    for aoi in response.json()["aois"]:
+        if aoi["aoi_name"] == area_name:
+            aoi_id = aoi["aoi_id"]
+            break
+
+    # Get all product IDs
+    params = {
+        "time": "range",
+        "from": f"{from_date}T00:00:00",
+        "to": f"{to_date}T00:00:00",
+    }
+    prod_url = f"{base_url}/aoi/{aoi_id}/products"
+    response = requests.get(prod_url, headers=header, params=params)
+    products = response.json()["products"]
+    print(f"Found {len(products)} products for {area_name}")
+
+    if not output_file_path:
+        base_file_path = "./output"
+
+    # Download all available flood products
+    for product in products:
+        product_id = product["product_id"]
+
+        # Converts product_time from e.g. "2025-01-05T06:10:37" to ""2025-01-05 06h
+        # Reason for bucketing per hour is that products are often seconds or minutes apart and should be grouped
+        product_time = product["product_time"]
+        product_time = product_time.split(":")[0].replace("T", " ") + "h"
+        output_file_path = f"{base_file_path}/{area_name}/{product_time}"
+        Path(output_file_path).mkdir(parents=True, exist_ok=True)
+
+        print(f"Downloading product: {product_id}")
+
+        download_url = f"{base_url}/download/product/{product_id}"
+        response = requests.get(download_url, headers=header)
+        download_link = response.json()["download_link"]
+
+        # Download and unzip file
+        r = requests.get(download_link)
+        with zipfile.ZipFile(io.BytesIO(r.content)) as z:
+            print("Extracting...")
+            z.extractall(str(Path(output_file_path)))
+
+    print("Done!")
+
+
+def get_existing_flood_geojson(area_name, product_time, output_file_path=None):
+    """
+    Getting a saved GFM flood geojson in an output folder of GFM files. Merge in one feature group if multiple.
+    """
+    product_time = product_time.replace(":", "_")
+    if not output_file_path:
+        output_file_path = f"./output/{area_name}/{product_time}"
+
+    # Combine multiple flood files into a FeatureGroup
+    flood_geojson_group = folium.FeatureGroup(name=f"{area_name} Floods {product_time}")
+
+    for flood_file in Path(output_file_path).glob("*FLOOD*.geojson"):
+        with open(flood_file, "r") as f:
+            geojson_data = json.load(f)
+            flood_layer = folium.GeoJson(geojson_data)
+            flood_geojson_group.add_child(flood_layer)
+
+    # TODO: consider merging multiple flood layers into one, to avoid overlap
+
+    return flood_geojson_group

app/src/utils.py

@@ -1,9 +1,11 @@
 """Functions for the layout of the Streamlit app, including the sidebar."""
+
 import base64
 import os
 from datetime import date
 
 import streamlit as st
+
 from src.config_parameters import params
 
 
@@ -92,88 +94,6 @@ def set_tool_page_style():
 
 
 # Sidebar
-@st.cache(allow_output_mutation=True)
-def get_base64_of_bin_file(png_file):
-    """
-    Get base64 from image file.
-
-    Inputs:
-        png_file (str): image filename
-
-    Returns:
-        str: encoded ASCII file
-    """
-    with open(png_file, "rb") as f:
-        data = f.read()
-    return base64.b64encode(data).decode()
-
-
-def build_markup_for_logo(
-    png_file,
-):
-    """
-    Create full string for navigation bar, including logo and title.
-
-    Inputs:
-        png_file (str): image filename
-        background_position (str): position logo
-        image_width (str): width logo
-        image_height (str): height logo
-
-    Returns
-        str: full string with logo and title for sidebar
-    """
-    binary_string = get_base64_of_bin_file(png_file)
-    return """
-            <style>
-                [data-testid="stSidebarNav"] {
-                    background-image: url("data:image/png;base64,%s");
-                    background-repeat: no-repeat;
-                    padding-top: 50px;
-                    padding-bottom: 10px;
-                    background-position: %s;
-                    background-size: %s %s;
-                }
-                [data-testid="stSidebarNav"]::before {
-                    content: "%s";
-                    margin-left: 20px;
-                    margin-top: 20px;
-                    margin-bottom: 20px;
-                    font-size: %s;
-                    font-weight: %s;
-                    position: relative;
-                    text-align: center;
-                    top: 85px;
-                }
-            </style>
-            """ % (
-        binary_string,
-        params["MA_logo_background_position"],
-        params["MA_logo_width"],
-        "",
-        params["sidebar_header"],
-        params["sidebar_header_fontsize"],
-        params["sidebar_header_fontweight"],
-    )
-
-
-def add_logo(png_file):
-    """
-    Add logo to sidebar.
-
-    Inputs:
-        png_file (str): image filename
-    Returns:
-        None
-    """
-    logo_markup = build_markup_for_logo(png_file)
-    # st.sidebar.title("ciao")
-    st.markdown(
-        logo_markup,
-        unsafe_allow_html=True,
-    )
-
-
 def add_about():
     """
     Add about and contacts to sidebar.
@@ -183,46 +103,16 @@ def add_about():
     Returns:
         None
     """
-    today = date.today().strftime("%B %d, %Y")
-
     # About textbox
     st.sidebar.markdown("## About")
     st.sidebar.markdown(
-        """
-        <div class='warning' style='
-            background-color: %s;
-            margin: 0px;
-            padding: 1em;'
-        '>
-            <p style='
-                margin-left:1em;
-                margin: 0px;
-                font-size: 1rem;
-                margin-bottom: 1em;
-            '>
-                    Last update: %s
-            </p>
-            <p style='
-                margin-left:1em;
-                font-size: 1rem;
-                margin: 0px
-            '>
-                <a href='%s'>
-                Wiki reference page</a><br>
-                <a href='%s'>
-                GitHub repository</a><br>
-                <a href='%s'>
-                Data Science Lab</a>
-            </p>
-        </div>
-        """
-        % (
-            params["about_box_background_color"],
-            today,
-            params["url_project_wiki"],
-            params["url_github_repo"],
-            params["url_data_science_wiki"],
-        ),
+        f"""
+        <p>
+            Todo: general about stuff <br />
+            <a href='{params["url_github_repo"]}'>
+            Github Repo</a>
+        </p>
+        """,
         unsafe_allow_html=True,
     )
 

app/src/utils_ee.py

@@ -1,36 +0,0 @@
-"""Module for ee-related functionalities."""
-import ee
-import streamlit as st
-from ee import oauth
-from google.oauth2 import service_account
-from src.utils import is_app_on_streamlit
-
-
-@st.experimental_memo
-def ee_initialize(force_use_service_account: bool = False):
-    """Initialise Google Earth Engine.
-
-    Checks whether the app is deployed on Streamlit Cloud and, based on the
-    result, initialises Google Earth Engine in different ways: if the app is
-    run locally, the credentials are retrieved from the user's credentials
-    stored in the local system (personal Google account is used). If the app
-    is deployed on Streamlit Cloud, credentials are taken from the secrets
-    field in the cloud (a dedicated service account is used).
-    Inputs:
-        force_use_service_account (bool): If True, the dedicated Google
-            service account is used, regardless of whether the app is run
-            locally or in the cloud. To be able to use a service account
-            locally, a file called "secrets.toml" should be added to the
-            folder ".streamlit", in the main project folder.
-
-    Returns:
-        None
-    """
-    if force_use_service_account or is_app_on_streamlit():
-        service_account_keys = st.secrets["ee_keys"]
-        credentials = service_account.Credentials.from_service_account_info(
-            service_account_keys, scopes=oauth.SCOPES
-        )
-        ee.Initialize(credentials)
-    else:
-        ee.Initialize()

app/src/utils_flood_analysis.py

@@ -1,369 +0,0 @@
-"""Functions to derive flood extent using Google Earth Engine."""
-import time
-
-import ee
-
-
-def _check_task_completed(task_id, verbose=False):
-    """
-    Return True if a task export completes successfully, else returns false.
-
-    Inputs:
-        task_id (str): Google Earth Engine task id
-
-    Returns:
-        boolean
-
-    """
-    status = ee.data.getTaskStatus(task_id)[0]
-    if status["state"] in (
-        ee.batch.Task.State.CANCELLED,
-        ee.batch.Task.State.FAILED,
-    ):
-        if "error_message" in status:
-            if verbose:
-                print(status["error_message"])
-        return True
-    elif status["state"] == ee.batch.Task.State.COMPLETED:
-        return True
-    return False
-
-
-def wait_for_tasks(task_ids, timeout=3600, verbose=False):
-    """
-    Wait for tasks to complete, fail, or timeout.
-
-    Wait for all active tasks if task_ids is not provided.
-    Note: Tasks will not be canceled after timeout, and
-    may continue to run.
-    Inputs:
-        task_ids (list):
-        timeout (int):
-
-    Returns:
-        None
-    """
-    start = time.time()
-    elapsed = 0
-    while elapsed < timeout or timeout == 0:
-        elapsed = time.time() - start
-        finished = [_check_task_completed(task) for task in task_ids]
-        if all(finished):
-            if verbose:
-                print(f"Tasks {task_ids} completed after {elapsed}s")
-            return True
-        time.sleep(5)
-    if verbose:
-        print(
-            f"Stopped waiting for {len(task_ids)} tasks \
-            after {timeout} seconds"
-        )
-    return False
-
-
-def export_flood_data(
-    flooded_area_vector,
-    flooded_area_raster,
-    image_before_flood,
-    image_after_flood,
-    region,
-    filename="flood_extents",
-    verbose=False,
-):
-    """
-    Export the results of derive_flood_extents function to Google Drive.
-
-    Inputs:
-        flooded_area_vector (ee.FeatureCollection): Detected flood extents as
-            vector geometries.
-        flooded_area_raster (ee.Image): Detected flood extents as a binary
-            raster.
-        image_before_flood (ee.Image): The 'before' Sentinel-1 image.
-        image_after_flood (ee.Image): The 'after' Sentinel-1 image containing
-            view of the flood waters.
-        region (ee.Geometry.Polygon): Geographic extent of analysis area.
-        filename (str): Desired filename prefix for exported files
-
-    Returns:
-        None
-    """
-    if verbose:
-        print(
-            "Exporting detected flood extents to your Google Drive. \
-            Please wait..."
-        )
-    s1_before_task = ee.batch.Export.image.toDrive(
-        image=image_before_flood,
-        description="export_before_s1_scene",
-        scale=30,
-        region=region,
-        fileNamePrefix=filename + "_s1_before",
-        crs="EPSG:4326",
-        fileFormat="GeoTIFF",
-    )
-
-    s1_after_task = ee.batch.Export.image.toDrive(
-        image=image_after_flood,
-        description="export_flooded_s1_scene",
-        scale=30,
-        region=region,
-        fileNamePrefix=filename + "_s1_after",
-        crs="EPSG:4326",
-        fileFormat="GeoTIFF",
-    )
-
-    raster_task = ee.batch.Export.image.toDrive(
-        image=flooded_area_raster,
-        description="export_flood_extents_raster",
-        scale=30,
-        region=region,
-        fileNamePrefix=filename + "_raster",
-        crs="EPSG:4326",
-        fileFormat="GeoTIFF",
-    )
-
-    vector_task = ee.batch.Export.table.toDrive(
-        collection=flooded_area_vector,
-        description="export_flood_extents_polygons",
-        fileFormat="shp",
-        fileNamePrefix=filename + "_polygons",
-    )
-
-    s1_before_task.start()
-    s1_after_task.start()
-    raster_task.start()
-    vector_task.start()
-
-    if verbose:
-        print("Exporting before Sentinel-1 scene: Task id ", s1_before_task.id)
-        print("Exporting flooded Sentinel-1 scene: Task id ", s1_after_task.id)
-        print("Exporting flood extent geotiff: Task id ", raster_task.id)
-        print("Exporting flood extent shapefile:  Task id ", vector_task.id)
-
-    wait_for_tasks(
-        [s1_before_task.id, s1_after_task.id, raster_task.id, vector_task.id]
-    )
-
-
-def retrieve_image_collection(
-    search_region,
-    start_date,
-    end_date,
-    polarization="VH",
-    pass_direction="Ascending",
-):
-    """
-    Retrieve Sentinel-1 immage collection from Google Earth Engine.
-
-    Inputs:
-        search_region (ee.Geometry.Polygon): Geographic extent of image search.
-        start_date (str): Date in format yyyy-mm-dd, e.g., '2020-10-01'.
-        end_date (str): Date in format yyyy-mm-dd, e.g., '2020-10-01'.
-        polarization (str): Synthetic aperture radar polarization mode, e.g.,
-            'VH' or 'VV'. VH is mostly is the preferred polarization for
-            flood mapping.
-        pass_direction (str): Synthetic aperture radar pass direction, either
-            'Ascending' or 'Descending'.
-
-    Returns:
-        collection (ee.ImageCollection): Sentinel-1 images matching the search
-        criteria.
-    """
-    collection = (
-        ee.ImageCollection("COPERNICUS/S1_GRD")
-        .filter(ee.Filter.eq("instrumentMode", "IW"))
-        .filter(
-            ee.Filter.listContains(
-                "transmitterReceiverPolarisation", polarization
-            )
-        )
-        .filter(ee.Filter.eq("orbitProperties_pass", pass_direction.upper()))
-        .filter(ee.Filter.eq("resolution_meters", 10))
-        .filterDate(start_date, end_date)
-        .filterBounds(search_region)
-        .select(polarization)
-    )
-
-    return collection
-
-
-def smooth(image, smoothing_radius=50):
-    """
-    Reduce the radar speckle by smoothing.
-
-    Inputs:
-        image (ee.Image): Input image.
-        smoothing_radius (int): The radius of the kernel to use for focal mean
-            smoothing.
-
-    Returns:
-        smoothed_image (ee.Image): The resulting image after smoothing is
-            applied.
-    """
-    smoothed_image = image.focal_mean(
-        radius=smoothing_radius, kernelType="circle", units="meters"
-    )
-
-    return smoothed_image
-
-
-def mask_permanent_water(image):
-    """
-    Query the JRC Global Surface Water Mapping Layers, v1.3.
-
-    The goal is to determine where perennial water bodies (water > 10
-    months/yr), and mask these areas.
-    Inputs:
-        image (ee.Image): Input image.
-
-    Returns:
-        masked_image (ee.Image): The resulting image after surface water
-        masking is applied.
-    """
-    surface_water = ee.Image("JRC/GSW1_4/GlobalSurfaceWater").select(
-        "seasonality"
-    )
-    surface_water_mask = surface_water.gte(10).updateMask(
-        surface_water.gte(10)
-    )
-
-    # Flooded layer where perennial water bodies(water > 10 mo / yr) is
-    # assigned a 0 value
-    where_surface_water = image.where(surface_water_mask, 0)
-
-    masked_image = image.updateMask(where_surface_water)
-
-    return masked_image
-
-
-def reduce_noise(image):
-    """
-    Reduce noise in the image.
-
-    Compute connectivity of pixels to eliminate those connected to 8 or fewer
-    neighbours.
-    Inputs:
-        image (ee.Image): A binary image.
-
-    Returns:
-        reduced_noise_image (ee.Image): The resulting image after noise
-            reduction is applied.
-    """
-    connections = image.connectedPixelCount()
-    reduced_noise_image = image.updateMask(connections.gte(8))
-
-    return reduced_noise_image
-
-
-def mask_slopes(image):
-    """
-    Mask out areas with more than 5 % slope with a Digital Elevation Model.
-
-    Inputs:
-        image (ee.Image): Input image.
-    Returns:
-         slopes_masked (ee.Image): The resulting image after slope masking is
-            applied.
-    """
-    dem = ee.Image("WWF/HydroSHEDS/03VFDEM")
-    terrain = ee.Algorithms.Terrain(dem)
-    slope = terrain.select("slope")
-    slopes_masked = image.updateMask(slope.lt(5))
-
-    return slopes_masked
-
-
-def derive_flood_extents(
-    aoi,
-    before_start_date,
-    before_end_date,
-    after_start_date,
-    after_end_date,
-    difference_threshold=1.25,
-    polarization="VH",
-    pass_direction="Ascending",
-    export=False,
-    export_filename="flood_extents",
-):
-    """
-    Set start and end dates of a period BEFORE and AFTER a flood.
-
-    These periods need to be long enough for Sentinel-1 to acquire an image.
-
-    Inputs:
-        aoi (ee.Geometry.Polygon): Geographic extent of analysis area.
-        before_start_date (str): Date in format yyyy-mm-dd, e.g., '2020-10-01'.
-        before_end_date (str): Date in format yyyy-mm-dd, e.g., '2020-10-01'.
-        after_start_date (str): Date in format yyyy-mm-dd, e.g., '2020-10-01'.
-        after_end_date (str): Date in format yyyy-mm-dd, e.g., '2020-10-01'.
-        difference_threshold (float): Threshold to be applied on the
-            differenced image (after flood - before flood). It has been chosen
-            by trial and error. In case your flood extent result shows many
-            false-positive or negative signals, consider changing it.
-        export (bool): Flag to export derived flood extents to Google Drive
-        export_filename (str): Desired filename prefix for exported files. Only
-            used if export=True.
-
-    Returns:
-        flood_vectors (ee.FeatureCollection): Detected flood extents as vector
-            geometries.
-        flood_rasters (ee.Image): Detected flood extents as a binary raster.
-        before_filtered (ee.Image): The 'before' Sentinel-1 image.
-        after_filtered (ee.Image): The 'after' Sentinel-1 image containing view
-            of the flood waters.
-    """
-    before_flood_img_col = retrieve_image_collection(
-        search_region=aoi,
-        start_date=before_start_date,
-        end_date=before_end_date,
-        polarization=polarization,
-        pass_direction=pass_direction,
-    )
-    after_flood_img_col = retrieve_image_collection(
-        search_region=aoi,
-        start_date=after_start_date,
-        end_date=after_end_date,
-        polarization=polarization,
-        pass_direction=pass_direction,
-    )
-
-    # Create a mosaic of selected tiles and clip to study area
-    before_mosaic = before_flood_img_col.mosaic().clip(aoi)
-    after_mosaic = after_flood_img_col.mosaic().clip(aoi)
-
-    before_filtered = smooth(before_mosaic)
-    after_filtered = smooth(after_mosaic)
-
-    # Calculate the difference between the before and after images
-    difference = after_filtered.divide(before_filtered)
-
-    # Apply the predefined difference - threshold and create the flood extent
-    # mask
-    difference_binary = difference.gt(difference_threshold)
-    difference_binary_masked = mask_permanent_water(difference_binary)
-    difference_binary_masked_reduced_noise = reduce_noise(
-        difference_binary_masked
-    )
-    flood_rasters = mask_slopes(difference_binary_masked_reduced_noise)
-
-    # Export the extent of detected flood in vector format
-    flood_vectors = flood_rasters.reduceToVectors(
-        scale=10,
-        geometryType="polygon",
-        geometry=aoi,
-        eightConnected=False,
-        bestEffort=True,
-        tileScale=2,
-    )
-
-    if export:
-        export_flood_data(
-            flooded_area_vector=flood_vectors,
-            flooded_area_raster=flood_rasters,
-            image_before_flood=before_filtered,
-            image_after_flood=after_filtered,
-            region=aoi,
-            filename=export_filename,
-        )
-
-    return flood_vectors, flood_rasters, before_filtered, after_filtered

bboxes/bboxes.json

@@ -0,0 +1,167 @@
+{
+    "Valencia": {
+        "bounding_box": {
+            "type": "Feature",
+            "properties": {},
+            "geometry": {
+                "type": "Polygon",
+                "coordinates": [
+                    [
+                        [
+                            -1.098633,
+                            39.227998
+                        ],
+                        [
+                            -1.098633,
+                            39.639538
+                        ],
+                        [
+                            0.439453,
+                            39.639538
+                        ],
+                        [
+                            0.439453,
+                            39.227998
+                        ],
+                        [
+                            -1.098633,
+                            39.227998
+                        ]
+                    ]
+                ]
+            }
+        }
+    },
+    "Tsjad": {
+        "bounding_box": {
+            "type": "Feature",
+            "properties": {},
+            "geometry": {
+                "type": "Polygon",
+                "coordinates": [
+                    [
+                        [
+                            17.050781,
+                            9.178025
+                        ],
+                        [
+                            17.050781,
+                            11.18918
+                        ],
+                        [
+                            20.313721,
+                            11.18918
+                        ],
+                        [
+                            20.313721,
+                            9.178025
+                        ],
+                        [
+                            17.050781,
+                            9.178025
+                        ]
+                    ]
+                ]
+            }
+        }
+    },
+    "Uganda": {
+        "bounding_box": {
+            "type": "Feature",
+            "properties": {},
+            "geometry": {
+                "type": "Polygon",
+                "coordinates": [
+                    [
+                        [
+                            32.519531,
+                            0.966751
+                        ],
+                        [
+                            32.519531,
+                            2.921097
+                        ],
+                        [
+                            34.145508,
+                            2.921097
+                        ],
+                        [
+                            34.145508,
+                            0.966751
+                        ],
+                        [
+                            32.519531,
+                            0.966751
+                        ]
+                    ]
+                ]
+            }
+        }
+    },
+    "Pego": {
+        "bounding_box": {
+            "type": "Feature",
+            "properties": {},
+            "geometry": {
+                "type": "Polygon",
+                "coordinates": [
+                    [
+                        [
+                            -0.085831,
+                            38.867781
+                        ],
+                        [
+                            -0.085831,
+                            38.884085
+                        ],
+                        [
+                            -0.058966,
+                            38.884085
+                        ],
+                        [
+                            -0.058966,
+                            38.867781
+                        ],
+                        [
+                            -0.085831,
+                            38.867781
+                        ]
+                    ]
+                ]
+            }
+        }
+    },
+    "Albufera": {
+        "bounding_box": {
+            "type": "Feature",
+            "properties": {},
+            "geometry": {
+                "type": "Polygon",
+                "coordinates": [
+                    [
+                        [
+                            -0.410614,
+                            39.2615
+                        ],
+                        [
+                            -0.410614,
+                            39.373057
+                        ],
+                        [
+                            -0.263672,
+                            39.373057
+                        ],
+                        [
+                            -0.263672,
+                            39.2615
+                        ],
+                        [
+                            -0.410614,
+                            39.2615
+                        ]
+                    ]
+                ]
+            }
+        }
+    }
+}

pyproject.toml

@@ -1,6 +1,15 @@
-[tool.black]
-line-length = 79
-
-[tool.isort]
-profile = "black"
-line_length = 79
+[project]
+name = "flood-mapping-gfm"
+version = "0.1.0"
+description = "A streamlit app to map gfm flood forecasts"
+readme = "README.md"
+requires-python = ">=3.12"
+dependencies = [
+    "requests>=2.32.3",
+    "geopandas>=1.0.1",
+    "folium>=0.19.4",
+    "python-dotenv==1.0.1",
+    "streamlit>=1.41.1",
+    "streamlit-folium>=0.24.0",
+    "ipykernel>=6.29.5",
+]

requirements.txt

@@ -1,7 +0,0 @@
-earthengine-api==0.1.331
-folium==0.13.0
-geemap==0.17.2
-streamlit==1.14.1
-streamlit_ext==0.1.4
-streamlit-folium==0.7.0
-pre-commit==2.18.1

setup.cfg

@@ -1,15 +0,0 @@
-[flake8]
-max-pos-args = 3
-ignore =
-    # Allow f-strings
-    SFS301,
-    # Allow print statements
-    T001,
-    # Allow implictly concatenated string literals in one line
-    ISC001,
-    # Allow percent operator in string
-    SFS101,
-    # Allow more than one # for comments
-    E266,
-    # Allow line break before binary operator
-    W503