stripped original app

.pre-commit-config.yaml

@@ -1,39 +0,0 @@
-default_language_version:
-    python: python3.9
-
-repos:
-  - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v4.4.0
-    hooks:
-      - id: trailing-whitespace
-      - id: end-of-file-fixer
-      - id: check-ast
-      - id: check-toml
-      - id: check-yaml
-  - repo: https://github.com/psf/black
-    rev: 23.1.0
-    hooks:
-      - id: black
-  - repo: https://github.com/PyCQA/isort
-    rev: 5.12.0
-    hooks:
-      - id: isort
-  - repo: https://github.com/pycqa/flake8
-    rev: 6.0.0
-    hooks:
-      - id: flake8
-        additional_dependencies:
-        #   - flake8-bugbear==21.9.2
-        #   - flake8-comprehensions==3.6.1
-          # - flake8-deprecated==1.3
-          - flake8-docstrings==1.6.0
-          # - flake8-keyword-arguments==0.1.0
-  - repo: https://github.com/pre-commit/mirrors-mypy
-    rev: v0.910
-    hooks:
-      - id: mypy
-        additional_dependencies:
-          - types-PyYAML==5.4.10
-          - types-setuptools==57.4.0
-          - types-requests==2.25.9
-        exclude: tests

README.md

@@ -1,63 +1,3 @@
 # Flood Mapping Tool
-
-[![Open in Streamlit](https://static.streamlit.io/badges/streamlit_badge_black_white.svg)](https://mapaction-flood-map.streamlit.app/)
-[![license](https://img.shields.io/github/license/OCHA-DAP/pa-aa-toolbox.svg)](https://github.com/mapaction/flood-mapping-tool/blob/main/LICENSE)
-[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
-[![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat&labelColor=ef8336)](https://pycqa.github.io/isort/)
-
-This repository contains a Streamlit app that allows to estimate flood extent using Sentinel-1 synthetic-aperture radar <a href='https://sentinel.esa.int/web/sentinel/user-guidessentinel-1-sar'>SAR</a> data.
-
-The methodology is based on a <a href='https://un-spider.org/advisory-support/recommended-practices/recommended-\practice-google-earth-engine-flood-mapping'> recommended practice </a> published by the United Nations Platform for Space-based Information for Disaster Management and Emergency Response (UN-SPIDER) and it uses several satellite imagery datasets to produce the final output. The datasets are retrieved from <a href='https://earthengine.google.com/'>Google Earth Engine</a> which is a powerful web-platform for cloud-based processing of remote sensing data on large scales. More information on the methodology is given in the <i>Description</i> page in the Streamlit app.
-
-This analysis provides a comprehensive overview of a flooding event, across different areas of interest, from settlements to countries. However, as mentioned in the UN-SPIDER website, the methodology is meant for broad information provision in a global context, and contains inherent uncertainties. Therefore, it is important that the tool is not used as the only source of information for rescue response planning.
-
-## Usage
-
-#### Requirements
-
-The Python version currently used is 3.10. Please install all packages from
-``requirements.txt``:
-
-```shell
-pip install -r requirements.txt
-```
-
-#### Google Earth Engine authentication
-
-[Sign up](https://signup.earthengine.google.com/) for a Google Earth Engine account, if you don't already have one. Open a terminal window, type `python` and then paste the following code:
-
-```python
-import ee
-ee.Authenticate()
-```
-
-Log in to your Google account to obtain the authorization code and paste it back into the terminal. Once you press "Enter", an authorization token will be saved to your computer under the following file path (depending on your operating system):
-
-- Windows: `C:\\Users\\USERNAME\\.config\\earthengine\\credentials`
-- Linux: `/home/USERNAME/.config/earthengine/credentials`
-- MacOS: `/Users/USERNAME/.config/earthengine/credentials`
-
-The credentials will be used when initialising Google Earth Engine in the app.
-
-#### Run the app
-
-Finally, open a terminal and run
-
-```shell
-streamlit run app/Home.py
-```
-
-A new browser window will open and you can start using the tool.
-
-## Contributing
-
-#### Pre-commit
-
-All code is formatted according to
-[black](https://github.com/psf/black) and [flake8](https://flake8.pycqa.org/en/latest) guidelines. The repo is set-up to use [pre-commit](https://github.com/pre-commit/pre-commit). Please run ``pre-commit install`` the first time you are editing. Thereafter all commits will be checked against black and flake8 guidelines.
-
-To check if your changes pass pre-commit without committing, run:
-
-```shell
-pre-commit run --all-files
-```
+TODO: New documentation. For now its kept on python 3.10 like original Mapaction app and packages have not yet been updated.
+Just create venv and pip install requirements

app/Home.py

@@ -1,4 +1,5 @@
 """Home page for Streamlit app."""
+
 import streamlit as st
 from src.config_parameters import params
 from src.utils import (
@@ -26,110 +27,8 @@ st.markdown("# Home")
 
 # First section
 st.markdown("## Introduction")
-st.markdown(
-    """
-    This tool allows to estimate flood extent using Sentinel-1
-    synthetic-aperture radar
-    <a href='%s'>SAR</a> data.<br><br>
-    The methodology is based on a <a href=
-    '%s'>recommended practice</a>
-    published by the United Nations Platform for Space-based Information for
-    Disaster Management and Emergency Response (UN-SPIDER) and it uses several
-    satellite imagery datasets to produce the final output. The datasets are
-    retrieved from <a href='%s'>Google Earth
-    Engine</a> which is a powerful web-platform for cloud-based processing of
-    remote sensing data on large scales. More information on the methodology is
-    given in the Description.<br><br>
-    This analysis provides a comprehensive overview of a flooding event, across
-    different areas of interest, from settlements to countries. However, as
-    mentioned in the UN-SPIDER website, the methodology is meant for broad
-    information provision in a global context, and contains inherent
-    uncertainties. Therefore, it is important that the tool is not used as the
-    only source of information for rescue response planning.
-    """
-    % (
-        params["url_sentinel_esa"],
-        params["url_unspider_tutorial"],
-        params["url_gee"],
-    ),
-    unsafe_allow_html=True,
-)
+st.markdown("TODO: new introduction")
 
 # Second section
 st.markdown("## How to use the tool")
-st.markdown(
-    """
-    <ul>
-        <li><p>
-            In the sidebar, choose <i>Flood extent analysis</i> to start the
-            analysis.
-        </p>
-        <li><p>
-            In the left panel, use the drawing tool to select an area of
-            interest on the map. You can delete your selection by clicking on
-            the bin icon. While the flood mapping is generated regardless of
-            the size of the selected region, you will be able to save raster
-            and vector flooding extent only if the side of the rectangular
-            selection does not exceed 100 km.
-        </p>
-        <li><p>
-            In the right panel click on the title <i>Choose Image Dates</i>
-            in order to expand the section. Here you need to select four dates.
-            The first two identify a range of dates based on which the
-            reference imagery (before the flooding event) is defined. You can
-            select even years worth of data (the reference imagery is
-            calculated as the median between the range of observations), but
-            make sure you take into account wet and dry seasons if only taking
-            a few months. The last two refer to a period of time which comes
-            after the flooding event. By setting periods, not single dates, you
-            allow the selection of enough tiles to cover the area of interest.
-            Sentinel-1 imagery is acquired minimum every 12 days for each point
-            on the globe (see Figure 2 in the documentation).
-        </p>
-        <li>
-            <p>
-                By clicking on <i>Choose parameters</i>, you will be able to
-                set two variables:
-            </p>
-            <ul>
-                <li><p>
-                    The <i>threshold</i> is the value against which the
-                    difference the two satellite images - before and after the
-                    flooding event - is tested. Lower thresholds result in a
-                    greater area considered "flooded". It is recommended to set
-                    the value to 1.25, which was selected through trial and
-                    error. You may want to adjust the value in case of high
-                    rates of false positive or negative values, especially in
-                    case other sources of information are available and it is
-                    possible to compare flood extent estimations between
-                    sources.
-                </p>
-                <li><p>
-                    The <i>pass direction</i> has to do with the way the
-                    satellite travels around the Earth. Depending on your area
-                    of interest and time period, you may find more imagery
-                    available for either the <i>Ascending</i> or the
-                    <i>Descending</i> pass directions (see Figure 2 in the
-                    Documentation). It is recommended to leave the parameter
-                    unchanged for a first estimation and change its value in
-                    case partial or no imagery is produced.
-                </p>
-            </ul>
-        <li><p>
-            Once the parameters are set, you can finally click on <i>Compute
-            flood extent</i> to run the calculations. A map will appear
-            underneath, with a layer containing the flooded area within the
-            area of interest.
-        </p>
-        <li><p>
-            If you wish to export the layer to file, you can click on <i>Export
-            to file</i> and download the raster and/or vector data.
-        </p>
-    </ul>
-    <p>
-        In case you get errors, follow the intructions. If you have doubts,
-        feel free to contact the Data Science team.
-    </p>
-    """,
-    unsafe_allow_html=True,
-)
+st.markdown("TODO: new how to use the tool")

app/pages/1_🌍_Flood_extent_analysis.py

@@ -1,7 +1,7 @@
 """Flood extent analysis page for Streamlit app."""
+
 import datetime as dt
 
-import ee
 import folium
 import geemap.foliumap as geemap
 import requests
@@ -15,7 +15,6 @@ from src.utils import (
     set_tool_page_style,
     toggle_menu_button,
 )
-from src.utils_ee import ee_initialize
 from src.utils_flood_analysis import derive_flood_extents
 from streamlit_folium import st_folium
 
@@ -35,10 +34,6 @@ st.markdown("# Flood extent analysis")
 # Set page style
 set_tool_page_style()
 
-# Initialise Google Earth Engine
-ee_initialize(force_use_service_account=True)
-
-
 # Output_created is useful to decide whether the bottom panel with the
 # output map should be visualised or not
 if "output_created" not in st.session_state:
@@ -73,8 +68,8 @@ with col1:
             draw_options={
                 "circle": False,
                 "polyline": False,
-                "polygon": True,
-                "circle": False,
+                "polygon": False,
+                "rectangle": True,
                 "marker": False,
                 "circlemarker": False,
             },
@@ -90,62 +85,24 @@ with col2:
     with st.expander("Choose Image Dates"):
         # Callback is added, so that, every time a parameters is changed,
         # the bottom panel containing the output map is hidden
-        before_start = st.date_input(
-            "Start date for reference imagery",
-            value=dt.date(year=2022, month=7, day=1),
-            help="It needs to be prior to the flooding event",
+        start_date = st.date_input(
+            "Start date",
             on_change=callback,
         )
-        before_end = st.date_input(
-            "End date for reference imagery",
-            value=dt.date(year=2022, month=7, day=30),
-            help=(
-                "It needs to be prior to the flooding event, at least 15 "
-                "days subsequent to the date selected above"
-            ),
-            on_change=callback,
-        )
-        after_start = st.date_input(
-            "Start date for flooding imagery",
-            value=dt.date(year=2022, month=9, day=1),
-            help="It needs to be subsequent to the flooding event",
-            on_change=callback,
-        )
-        after_end = st.date_input(
-            "End date for flooding imagery",
-            value=dt.date(year=2022, month=9, day=16),
-            help=(
-                "It needs to be subsequent to the flooding event and at "
-                "least 10 days to the date selected above"
-            ),
+        end_date = st.date_input(
+            "End date",
             on_change=callback,
         )
     # Add collapsable container for parameters
     with st.expander("Choose Parameters"):
         # Add slider for threshold
-        add_slider = st.slider(
-            label="Select a threshold",
-            min_value=0.0,
-            max_value=5.0,
-            value=1.25,
-            step=0.25,
-            help="Higher values might reduce overall noise",
-            on_change=callback,
-        )
-        # Add radio buttons for pass direction
-        pass_direction = st.radio(
-            "Set pass direction",
-            ["Ascending", "Descending"],
-            on_change=callback,
-        )
+        st.text("Add relevant (API) parameters here")
     # Button for computation
-    submitted = st.button("Compute flood extent")
+    submitted = st.button("Get flood extent")
     # Introduce date validation
-    check_dates = before_start < before_end <= after_start < after_end
+    check_dates = start_date <= end_date
     # Introduce drawing validation (a polygon needs to exist)
-    check_drawing = (
-        output["all_drawings"] != [] and output["all_drawings"] is not None
-    )
+    check_drawing = output["all_drawings"] != [] and output["all_drawings"] is not None
 # What happens when button is clicked on?
 if submitted:
     with col2:
@@ -159,144 +116,70 @@ if submitted:
             # Add output for computation
             with st.spinner("Computing... Please wait..."):
                 # Extract coordinates from drawn polygon
-                coords = output["all_drawings"][-1]["geometry"]["coordinates"][
-                    0
-                ]
+                coords = output["all_drawings"][-1]["geometry"]["coordinates"][0]
+                print(f"Coords: {coords}")
                 # Create geometry from coordinates
-                ee_geom_region = ee.Geometry.Polygon(coords)
-                # Crate flood raster and vector
-                (
-                    detected_flood_vector,
-                    detected_flood_raster,
-                    _,
-                    _,
-                ) = derive_flood_extents(
-                    aoi=ee_geom_region,
-                    before_start_date=str(before_start),
-                    before_end_date=str(before_end),
-                    after_start_date=str(after_start),
-                    after_end_date=str(after_end),
-                    difference_threshold=add_slider,
-                    polarization="VH",
-                    pass_direction=pass_direction,
-                    export=False,
-                )
-                # Create output map
-                Map2 = geemap.Map(
-                    # basemap="HYBRID",
-                    plugin_Draw=False,
-                    Draw_export=False,
-                    locate_control=False,
-                    plugin_LatLngPopup=False,
-                )
-                try:
-                    # Add flood vector layer to map
-                    Map2.add_layer(
-                        ee_object=detected_flood_raster,
-                        name="Flood extent raster",
-                    )
-                    Map2.add_layer(
-                        ee_object=detected_flood_vector,
-                        name="Flood extent vector",
-                    )
-                    # Center map on flood raster
-                    Map2.centerObject(detected_flood_raster)
-                except ee.EEException:
-                    # If error contains the sentence below, it means that
-                    # an image could not be properly generated
-                    st.error(
-                        """
-                        No satellite image found for the selected
-                        dates.\n\n
-                        Try changing the pass direction.\n\n
-                        If this does not work, choose different
-                        dates: it is likely that the satellite did not
-                        cover the area of interest in the range of
-                        dates specified (either before or after the
-                        flooding event).
-                        """
-                    )
-                else:
-                    # If computation was succesfull, save outputs for
-                    # output map
-                    st.success("Computation complete")
-                    st.session_state.output_created = True
-                    st.session_state.Map2 = Map2
-                    st.session_state.detected_flood_raster = (
-                        detected_flood_raster
-                    )
-                    st.session_state.detected_flood_vector = (
-                        detected_flood_vector
-                    )
-                    st.session_state.ee_geom_region = ee_geom_region
+                st.session_state.output_created = True
+
 # If computation was successful, create output map in bottom panel
 if st.session_state.output_created:
     with row2:
         # Add collapsable container for output map
         with st.expander("Output map", expanded=True):
-            # Export Map2 to streamlit
-            st.session_state.Map2.to_streamlit()
-            # Create button to export to file
-            submitted2 = st.button("Export to file")
-            # What happens if button is clicked on?
-            if submitted2:
-                # Add output for computation
-                with st.spinner("Computing... Please wait..."):
-                    try:
-                        # Get download url for raster data
-                        raster = st.session_state.detected_flood_raster
-                        url_r = raster.getDownloadUrl(
-                            {
-                                "region": st.session_state.ee_geom_region,
-                                "scale": 30,
-                                "format": "GEO_TIFF",
-                            }
-                        )
-                    except Exception:
-                        st.error(
-                            """
-                            The image size is too big for the image to
-                            be exported to file. Select a smaller area
-                            of interest (side <~ 150km) and repeat the
-                            analysis.
-                            """
-                        )
-                    else:
-                        response_r = requests.get(url_r)
-                        # Get download url for raster data
-                        vector = st.session_state.detected_flood_vector
-                        url_v = vector.getDownloadUrl("GEOJSON")
-                        response_v = requests.get(url_v)
-                        filename = "flood_extent"
-                        timestamp = dt.datetime.now().strftime(
-                            "%Y-%m-%d_%H-%M"
-                        )
-                        with row2:
-                            # Create download buttons for raster and vector
-                            # data
-                            with open("flood_extent.tif", "wb"):
-                                ste.download_button(
-                                    label="Download Raster Extent",
-                                    data=response_r.content,
-                                    file_name=(
-                                        f"{filename}"
-                                        "_raster_"
-                                        f"{timestamp}"
-                                        ".tif"
-                                    ),
-                                    mime="image/tif",
-                                )
-                            with open("flood_extent.geojson", "wb"):
-                                ste.download_button(
-                                    label="Download Vector Extent",
-                                    data=response_v.content,
-                                    file_name=(
-                                        f"{filename}"
-                                        "_vector_"
-                                        f"{timestamp}"
-                                        ".geojson"
-                                    ),
-                                    mime="text/json",
-                                )
-                        # Output for computation complete
-                        st.success("Computation complete")
+            st.success("Calc complete")
+            # # Export Map2 to streamlit
+            # st.session_state.Map2.to_streamlit()
+            # # Create button to export to file
+            # submitted2 = st.button("Export to file")
+            # # What happens if button is clicked on?
+            # if submitted2:
+            #     # Add output for computation
+            #     with st.spinner("Computing... Please wait..."):
+            #         try:
+            #             # Get download url for raster data
+            #             raster = st.session_state.detected_flood_raster
+            #             url_r = raster.getDownloadUrl(
+            #                 {
+            #                     "region": st.session_state.ee_geom_region,
+            #                     "scale": 30,
+            #                     "format": "GEO_TIFF",
+            #                 }
+            #             )
+            #         except Exception:
+            #             st.error(
+            #                 """
+            #                 The image size is too big for the image to
+            #                 be exported to file. Select a smaller area
+            #                 of interest (side <~ 150km) and repeat the
+            #                 analysis.
+            #                 """
+            #             )
+            #         else:
+            #             response_r = requests.get(url_r)
+            #             # Get download url for raster data
+            #             vector = st.session_state.detected_flood_vector
+            #             url_v = vector.getDownloadUrl("GEOJSON")
+            #             response_v = requests.get(url_v)
+            #             filename = "flood_extent"
+            #             timestamp = dt.datetime.now().strftime("%Y-%m-%d_%H-%M")
+            #             with row2:
+            #                 # Create download buttons for raster and vector
+            #                 # data
+            #                 with open("flood_extent.tif", "wb"):
+            #                     ste.download_button(
+            #                         label="Download Raster Extent",
+            #                         data=response_r.content,
+            #                         file_name=(f"{filename}_raster_{timestamp}.tif"),
+            #                         mime="image/tif",
+            #                     )
+            #                 with open("flood_extent.geojson", "wb"):
+            #                     ste.download_button(
+            #                         label="Download Vector Extent",
+            #                         data=response_v.content,
+            #                         file_name=(
+            #                             f"{filename}_vector_{timestamp}.geojson"
+            #                         ),
+            #                         mime="text/json",
+            #                     )
+            #             # Output for computation complete
+            #             st.success("Computation complete")

app/pages/2_📖_Documentation.py

@@ -1,4 +1,5 @@
 """Documentation page for Streamlit app."""
+
 import streamlit as st
 from PIL import Image
 from src.config_parameters import params
@@ -28,63 +29,9 @@ st.markdown("# Documentation")
 # First section
 st.markdown("## Methodology")
 st.markdown(
-    """
-    The methodology is based on the workflow depicted in Figure 1. In
-    addition to Sentinel-1 synthetic-aperture radar <a href='%s'>SAR</a> data,
-    two other datasets are used through <a href='%s'>Google Earth Engine</a>:
-    <ul>
-        <li><p>
-            The <i>WWF HydroSHEDS Void-Filled DEM, 3 Arc-Seconds</i>
-            <a href='%s'>dataset</a> is based on elevation data
-            obtained in 2000 by NASA's Shuttle Radar Topography Mission (SRTM),
-            and it is used to mask out areas with more than 5 percent slope
-            (see following section on limitations).
-        </p>
-        <li><p>
-            The <i>JRC Global Surface Water Mapping Layers, v1.4</i>
-            <a href='%s'>dataset</a> contains maps of the
-            location and temporal distribution of surface water from 1984 to
-            2021, and it is used to mask areas with perennial water bodies,
-            such as rivers or lakes.
-        </p>
-    </ul>
-    """
-    % (
-        params["url_sentinel_dataset"],
-        params["url_gee"],
-        params["url_elevation_dataset"],
-        params["url_surface_water_dataset"],
-    ),
-    unsafe_allow_html=True,
+    "TODO: new documentation, only kept in Sentinel 1 section unchanged from the Mapaction tool"
 )
 
-# Add image workflow
-img = Image.open("app/img/workflow.png")
-col1, mid, col2, last = st.columns([5, 3, 10, 10])
-with col1:
-    st.image(img, width=350)
-with col2:
-    # Trick to add caption at the bottom of the column, as Streamlit has not
-    # developed a functionality to allign text to bottom
-    space_before_caption = "<br>" * 27
-    st.markdown(
-        space_before_caption,
-        unsafe_allow_html=True,
-    )
-    st.markdown(
-        """
-        <p style="font-size:%s;">
-            Figure 1. Workflow of the flood mapping methodology (<a href=
-            '%s'>source</a>).
-        </p>
-        """
-        % (
-            params["docs_caption_fontsize"],
-            params["url_unspider_tutorial_detail"],
-        ),
-        unsafe_allow_html=True,
-    )
-
 
 # Second section
 st.markdown("## Radar imagery for flood detection")
@@ -140,40 +87,3 @@ st.markdown(
     % (params["docs_caption_fontsize"], params["url_sentinel_img_location"]),
     unsafe_allow_html=True,
 )
-
-# Third section
-st.markdown("## Key limitations")
-st.markdown(
-    """
-    Radar imagery is great for detecting floods, as it is good at picking up
-    water and it is not affected by the time of the day or clouds (at this
-    wavelength). But it has its limits, and performs actually quite bad if
-    having to detect water in mountainous regions, especially if with narrow
-    valleys, and in urban areas (urban canyons). The reasons are mainly around
-    the viewing angles, which can cause image distortions. This method may also
-    result in false positives for other land cover changes with smooth
-    surfaces, such as roads and sand. Rough surface texture caused by wind or
-    rainfall may also make it challenging for the radar imagery to identify
-    water bodies.
-    """,
-    unsafe_allow_html=True,
-)
-
-
-# Last section
-st.markdown("## Useful links")
-st.markdown(
-    """
-    <a href='%s'>UN-SPIDER recommended practice</a><br>
-    <a href='%s'>Sentinel-1 satellite imagery user guide</a><br>
-    Relevant scientific publications:
-    <a href='%s'>1</a>, <a href='%s'>2</a><br>
-    """
-    % (
-        params["url_unspider_tutorial"],
-        params["url_sentinel_esa"],
-        params["url_publication_1"],
-        params["url_publication_2"],
-    ),
-    unsafe_allow_html=True,
-)

app/src/config_parameters.py

@@ -1,60 +1,23 @@
 """Configuration file."""
+
 params = {
     # Title browser tab
-    "browser_title": "Flood mapping tool - MapAction",
+    "browser_title": "Flood mapping tool - 510",
     # Data scientists involved
     "data_scientists": {
-        "Piet": "pgerrits@mapaction.org",
-        "Daniele": "[email protected]",
-        "Cate": "[email protected]",
+        "Daniele": "dcastellana@redcross.nl",
     },
     # Urls
-    "url_data_science_wiki": (
-        "https://mapaction.atlassian.net/wiki/spaces/GAFO/overview"
-    ),
-    "url_gee": "https://earthengine.google.com/",
-    "url_project_wiki": (
-        "https://mapaction.atlassian.net/wiki/spaces/GAFO/pages/15920922751/"
-        "Rapid+flood+mapping+from+satellite+imagery"
-    ),
     "url_github_repo": "https://github.com/mapaction/flood-extent-tool",
-    "url_sentinel_esa": (
-        "https://sentinel.esa.int/web/sentinel/user-guides/sentinel-1-sar"
-    ),
     "url_sentinel_dataset": (
-        "https://developers.google.com/earth-engine/datasets/catalog/"
-        "COPERNICUS_S1_GRD"
+        "https://developers.google.com/earth-engine/datasets/catalog/COPERNICUS_S1_GRD"
     ),
     "url_sentinel_img": (
         "https://sentinel.esa.int/documents/247904/4748961/Sentinel-1-Repeat-"
         "Coverage-Frequency-Geometry-2021.jpg"
     ),
     "url_sentinel_img_location": (
-        "https://sentinel.esa.int/web/sentinel/missions/sentinel-1/"
-        "observation-scenario"
-    ),
-    "url_unspider_tutorial": (
-        "https://un-spider.org/advisory-support/recommended-practices/"
-        "recommended-practice-google-earth-engine-flood-mapping"
-    ),
-    "url_unspider_tutorial_detail": (
-        "https://un-spider.org/advisory-support/recommended-practices/"
-        "recommended-practice-google-earth-engine-flood-mapping/in-detail"
-    ),
-    "url_elevation_dataset": (
-        "https://developers.google.com/earth-engine/datasets/catalog/"
-        "WWF_HydroSHEDS_03VFDEM"
-    ),
-    "url_surface_water_dataset": (
-        "https://developers.google.com/earth-engine/datasets/catalog/"
-        "JRC_GSW1_4_GlobalSurfaceWater"
-    ),
-    "url_publication_1": (
-        "https://onlinelibrary.wiley.com/doi/full/10.1111/jfr3.12303"
-    ),
-    "url_publication_2": (
-        "https://www.sciencedirect.com/science/article/abs/pii/"
-        "S0924271620301702"
+        "https://sentinel.esa.int/web/sentinel/missions/sentinel-1/observation-scenario"
     ),
     # Layout and styles
     ## Sidebar

app/src/utils.py

@@ -1,9 +1,11 @@
 """Functions for the layout of the Streamlit app, including the sidebar."""
+
 import base64
 import os
 from datetime import date
 
 import streamlit as st
+
 from src.config_parameters import params
 
 
@@ -183,46 +185,16 @@ def add_about():
     Returns:
         None
     """
-    today = date.today().strftime("%B %d, %Y")
-
     # About textbox
     st.sidebar.markdown("## About")
     st.sidebar.markdown(
-        """
-        <div class='warning' style='
-            background-color: %s;
-            margin: 0px;
-            padding: 1em;'
-        '>
-            <p style='
-                margin-left:1em;
-                margin: 0px;
-                font-size: 1rem;
-                margin-bottom: 1em;
-            '>
-                    Last update: %s
-            </p>
-            <p style='
-                margin-left:1em;
-                font-size: 1rem;
-                margin: 0px
-            '>
-                <a href='%s'>
-                Wiki reference page</a><br>
-                <a href='%s'>
-                GitHub repository</a><br>
-                <a href='%s'>
-                Data Science Lab</a>
-            </p>
-        </div>
-        """
-        % (
-            params["about_box_background_color"],
-            today,
-            params["url_project_wiki"],
-            params["url_github_repo"],
-            params["url_data_science_wiki"],
-        ),
+        f"""
+        <p>
+            Todo: general about stuff <br />
+            <a href='{params["url_github_repo"]}'>
+            Github Repo</a>
+        </p>
+        """,
         unsafe_allow_html=True,
     )
 

app/src/utils_ee.py

@@ -1,36 +0,0 @@
-"""Module for ee-related functionalities."""
-import ee
-import streamlit as st
-from ee import oauth
-from google.oauth2 import service_account
-from src.utils import is_app_on_streamlit
-
-
-@st.experimental_memo
-def ee_initialize(force_use_service_account: bool = False):
-    """Initialise Google Earth Engine.
-
-    Checks whether the app is deployed on Streamlit Cloud and, based on the
-    result, initialises Google Earth Engine in different ways: if the app is
-    run locally, the credentials are retrieved from the user's credentials
-    stored in the local system (personal Google account is used). If the app
-    is deployed on Streamlit Cloud, credentials are taken from the secrets
-    field in the cloud (a dedicated service account is used).
-    Inputs:
-        force_use_service_account (bool): If True, the dedicated Google
-            service account is used, regardless of whether the app is run
-            locally or in the cloud. To be able to use a service account
-            locally, a file called "secrets.toml" should be added to the
-            folder ".streamlit", in the main project folder.
-
-    Returns:
-        None
-    """
-    if force_use_service_account or is_app_on_streamlit():
-        service_account_keys = st.secrets["ee_keys"]
-        credentials = service_account.Credentials.from_service_account_info(
-            service_account_keys, scopes=oauth.SCOPES
-        )
-        ee.Initialize(credentials)
-    else:
-        ee.Initialize()

app/src/utils_flood_analysis.py

@@ -1,369 +0,0 @@
-"""Functions to derive flood extent using Google Earth Engine."""
-import time
-
-import ee
-
-
-def _check_task_completed(task_id, verbose=False):
-    """
-    Return True if a task export completes successfully, else returns false.
-
-    Inputs:
-        task_id (str): Google Earth Engine task id
-
-    Returns:
-        boolean
-
-    """
-    status = ee.data.getTaskStatus(task_id)[0]
-    if status["state"] in (
-        ee.batch.Task.State.CANCELLED,
-        ee.batch.Task.State.FAILED,
-    ):
-        if "error_message" in status:
-            if verbose:
-                print(status["error_message"])
-        return True
-    elif status["state"] == ee.batch.Task.State.COMPLETED:
-        return True
-    return False
-
-
-def wait_for_tasks(task_ids, timeout=3600, verbose=False):
-    """
-    Wait for tasks to complete, fail, or timeout.
-
-    Wait for all active tasks if task_ids is not provided.
-    Note: Tasks will not be canceled after timeout, and
-    may continue to run.
-    Inputs:
-        task_ids (list):
-        timeout (int):
-
-    Returns:
-        None
-    """
-    start = time.time()
-    elapsed = 0
-    while elapsed < timeout or timeout == 0:
-        elapsed = time.time() - start
-        finished = [_check_task_completed(task) for task in task_ids]
-        if all(finished):
-            if verbose:
-                print(f"Tasks {task_ids} completed after {elapsed}s")
-            return True
-        time.sleep(5)
-    if verbose:
-        print(
-            f"Stopped waiting for {len(task_ids)} tasks \
-            after {timeout} seconds"
-        )
-    return False
-
-
-def export_flood_data(
-    flooded_area_vector,
-    flooded_area_raster,
-    image_before_flood,
-    image_after_flood,
-    region,
-    filename="flood_extents",
-    verbose=False,
-):
-    """
-    Export the results of derive_flood_extents function to Google Drive.
-
-    Inputs:
-        flooded_area_vector (ee.FeatureCollection): Detected flood extents as
-            vector geometries.
-        flooded_area_raster (ee.Image): Detected flood extents as a binary
-            raster.
-        image_before_flood (ee.Image): The 'before' Sentinel-1 image.
-        image_after_flood (ee.Image): The 'after' Sentinel-1 image containing
-            view of the flood waters.
-        region (ee.Geometry.Polygon): Geographic extent of analysis area.
-        filename (str): Desired filename prefix for exported files
-
-    Returns:
-        None
-    """
-    if verbose:
-        print(
-            "Exporting detected flood extents to your Google Drive. \
-            Please wait..."
-        )
-    s1_before_task = ee.batch.Export.image.toDrive(
-        image=image_before_flood,
-        description="export_before_s1_scene",
-        scale=30,
-        region=region,
-        fileNamePrefix=filename + "_s1_before",
-        crs="EPSG:4326",
-        fileFormat="GeoTIFF",
-    )
-
-    s1_after_task = ee.batch.Export.image.toDrive(
-        image=image_after_flood,
-        description="export_flooded_s1_scene",
-        scale=30,
-        region=region,
-        fileNamePrefix=filename + "_s1_after",
-        crs="EPSG:4326",
-        fileFormat="GeoTIFF",
-    )
-
-    raster_task = ee.batch.Export.image.toDrive(
-        image=flooded_area_raster,
-        description="export_flood_extents_raster",
-        scale=30,
-        region=region,
-        fileNamePrefix=filename + "_raster",
-        crs="EPSG:4326",
-        fileFormat="GeoTIFF",
-    )
-
-    vector_task = ee.batch.Export.table.toDrive(
-        collection=flooded_area_vector,
-        description="export_flood_extents_polygons",
-        fileFormat="shp",
-        fileNamePrefix=filename + "_polygons",
-    )
-
-    s1_before_task.start()
-    s1_after_task.start()
-    raster_task.start()
-    vector_task.start()
-
-    if verbose:
-        print("Exporting before Sentinel-1 scene: Task id ", s1_before_task.id)
-        print("Exporting flooded Sentinel-1 scene: Task id ", s1_after_task.id)
-        print("Exporting flood extent geotiff: Task id ", raster_task.id)
-        print("Exporting flood extent shapefile:  Task id ", vector_task.id)
-
-    wait_for_tasks(
-        [s1_before_task.id, s1_after_task.id, raster_task.id, vector_task.id]
-    )
-
-
-def retrieve_image_collection(
-    search_region,
-    start_date,
-    end_date,
-    polarization="VH",
-    pass_direction="Ascending",
-):
-    """
-    Retrieve Sentinel-1 immage collection from Google Earth Engine.
-
-    Inputs:
-        search_region (ee.Geometry.Polygon): Geographic extent of image search.
-        start_date (str): Date in format yyyy-mm-dd, e.g., '2020-10-01'.
-        end_date (str): Date in format yyyy-mm-dd, e.g., '2020-10-01'.
-        polarization (str): Synthetic aperture radar polarization mode, e.g.,
-            'VH' or 'VV'. VH is mostly is the preferred polarization for
-            flood mapping.
-        pass_direction (str): Synthetic aperture radar pass direction, either
-            'Ascending' or 'Descending'.
-
-    Returns:
-        collection (ee.ImageCollection): Sentinel-1 images matching the search
-        criteria.
-    """
-    collection = (
-        ee.ImageCollection("COPERNICUS/S1_GRD")
-        .filter(ee.Filter.eq("instrumentMode", "IW"))
-        .filter(
-            ee.Filter.listContains(
-                "transmitterReceiverPolarisation", polarization
-            )
-        )
-        .filter(ee.Filter.eq("orbitProperties_pass", pass_direction.upper()))
-        .filter(ee.Filter.eq("resolution_meters", 10))
-        .filterDate(start_date, end_date)
-        .filterBounds(search_region)
-        .select(polarization)
-    )
-
-    return collection
-
-
-def smooth(image, smoothing_radius=50):
-    """
-    Reduce the radar speckle by smoothing.
-
-    Inputs:
-        image (ee.Image): Input image.
-        smoothing_radius (int): The radius of the kernel to use for focal mean
-            smoothing.
-
-    Returns:
-        smoothed_image (ee.Image): The resulting image after smoothing is
-            applied.
-    """
-    smoothed_image = image.focal_mean(
-        radius=smoothing_radius, kernelType="circle", units="meters"
-    )
-
-    return smoothed_image
-
-
-def mask_permanent_water(image):
-    """
-    Query the JRC Global Surface Water Mapping Layers, v1.3.
-
-    The goal is to determine where perennial water bodies (water > 10
-    months/yr), and mask these areas.
-    Inputs:
-        image (ee.Image): Input image.
-
-    Returns:
-        masked_image (ee.Image): The resulting image after surface water
-        masking is applied.
-    """
-    surface_water = ee.Image("JRC/GSW1_4/GlobalSurfaceWater").select(
-        "seasonality"
-    )
-    surface_water_mask = surface_water.gte(10).updateMask(
-        surface_water.gte(10)
-    )
-
-    # Flooded layer where perennial water bodies(water > 10 mo / yr) is
-    # assigned a 0 value
-    where_surface_water = image.where(surface_water_mask, 0)
-
-    masked_image = image.updateMask(where_surface_water)
-
-    return masked_image
-
-
-def reduce_noise(image):
-    """
-    Reduce noise in the image.
-
-    Compute connectivity of pixels to eliminate those connected to 8 or fewer
-    neighbours.
-    Inputs:
-        image (ee.Image): A binary image.
-
-    Returns:
-        reduced_noise_image (ee.Image): The resulting image after noise
-            reduction is applied.
-    """
-    connections = image.connectedPixelCount()
-    reduced_noise_image = image.updateMask(connections.gte(8))
-
-    return reduced_noise_image
-
-
-def mask_slopes(image):
-    """
-    Mask out areas with more than 5 % slope with a Digital Elevation Model.
-
-    Inputs:
-        image (ee.Image): Input image.
-    Returns:
-         slopes_masked (ee.Image): The resulting image after slope masking is
-            applied.
-    """
-    dem = ee.Image("WWF/HydroSHEDS/03VFDEM")
-    terrain = ee.Algorithms.Terrain(dem)
-    slope = terrain.select("slope")
-    slopes_masked = image.updateMask(slope.lt(5))
-
-    return slopes_masked
-
-
-def derive_flood_extents(
-    aoi,
-    before_start_date,
-    before_end_date,
-    after_start_date,
-    after_end_date,
-    difference_threshold=1.25,
-    polarization="VH",
-    pass_direction="Ascending",
-    export=False,
-    export_filename="flood_extents",
-):
-    """
-    Set start and end dates of a period BEFORE and AFTER a flood.
-
-    These periods need to be long enough for Sentinel-1 to acquire an image.
-
-    Inputs:
-        aoi (ee.Geometry.Polygon): Geographic extent of analysis area.
-        before_start_date (str): Date in format yyyy-mm-dd, e.g., '2020-10-01'.
-        before_end_date (str): Date in format yyyy-mm-dd, e.g., '2020-10-01'.
-        after_start_date (str): Date in format yyyy-mm-dd, e.g., '2020-10-01'.
-        after_end_date (str): Date in format yyyy-mm-dd, e.g., '2020-10-01'.
-        difference_threshold (float): Threshold to be applied on the
-            differenced image (after flood - before flood). It has been chosen
-            by trial and error. In case your flood extent result shows many
-            false-positive or negative signals, consider changing it.
-        export (bool): Flag to export derived flood extents to Google Drive
-        export_filename (str): Desired filename prefix for exported files. Only
-            used if export=True.
-
-    Returns:
-        flood_vectors (ee.FeatureCollection): Detected flood extents as vector
-            geometries.
-        flood_rasters (ee.Image): Detected flood extents as a binary raster.
-        before_filtered (ee.Image): The 'before' Sentinel-1 image.
-        after_filtered (ee.Image): The 'after' Sentinel-1 image containing view
-            of the flood waters.
-    """
-    before_flood_img_col = retrieve_image_collection(
-        search_region=aoi,
-        start_date=before_start_date,
-        end_date=before_end_date,
-        polarization=polarization,
-        pass_direction=pass_direction,
-    )
-    after_flood_img_col = retrieve_image_collection(
-        search_region=aoi,
-        start_date=after_start_date,
-        end_date=after_end_date,
-        polarization=polarization,
-        pass_direction=pass_direction,
-    )
-
-    # Create a mosaic of selected tiles and clip to study area
-    before_mosaic = before_flood_img_col.mosaic().clip(aoi)
-    after_mosaic = after_flood_img_col.mosaic().clip(aoi)
-
-    before_filtered = smooth(before_mosaic)
-    after_filtered = smooth(after_mosaic)
-
-    # Calculate the difference between the before and after images
-    difference = after_filtered.divide(before_filtered)
-
-    # Apply the predefined difference - threshold and create the flood extent
-    # mask
-    difference_binary = difference.gt(difference_threshold)
-    difference_binary_masked = mask_permanent_water(difference_binary)
-    difference_binary_masked_reduced_noise = reduce_noise(
-        difference_binary_masked
-    )
-    flood_rasters = mask_slopes(difference_binary_masked_reduced_noise)
-
-    # Export the extent of detected flood in vector format
-    flood_vectors = flood_rasters.reduceToVectors(
-        scale=10,
-        geometryType="polygon",
-        geometry=aoi,
-        eightConnected=False,
-        bestEffort=True,
-        tileScale=2,
-    )
-
-    if export:
-        export_flood_data(
-            flooded_area_vector=flood_vectors,
-            flooded_area_raster=flood_rasters,
-            image_before_flood=before_filtered,
-            image_after_flood=after_filtered,
-            region=aoi,
-            filename=export_filename,
-        )
-
-    return flood_vectors, flood_rasters, before_filtered, after_filtered

requirements.txt

@@ -5,3 +5,4 @@ streamlit==1.14.1
 streamlit_ext==0.1.4
 streamlit-folium==0.7.0
 pre-commit==2.18.1
+altair<5