initial commit without binaries or large files for huggingface

.gitignore

@@ -0,0 +1,16 @@
+*.wav
+*.zip
+utils/__*
+*__pycache__*
+acodet/_*
+acodet/src/tmp_*
+acodet/src/models/Humpback_20221130/
+_*
+.vscode/*
+tests/test_files/analysi*
+tests/test_files/test_annoted_files/thresh_0.5/N1/analysi*
+generated*
+tests/test_files/test_gen*
+tests/test_files/test_com*
+tests/test_files/test_tfr*
+simple_test*

.pre-commit-config.yaml

@@ -0,0 +1,6 @@
+repos:
+-   repo: https://github.com/psf/black
+    rev: 23.7.0
+    hooks:
+    - id: black
+      language_version: python3.9

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) [2022] [Vincent Kather]
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,328 @@
+# **acodet** - **Aco**ustic **Det**ector
+## Framework for the **usage** and **training** of acoustic species detectors based on CNN models
+
+### Highlights
+- **Integrated graphical user interface (GUI), so no coding required!**
+- Supports Raven table format
+    - resulting spreadsheets can be directly imported into raven to view annotations
+- automatic generation of presence/absence visualizations
+    - GUI supports interactive visualizations, allowing you to adjust model thresholds and instantly view the results
+- headless version included for those that prefer command line tools
+- interactive post-processing methods to reduce false-positives
+---------------------------------------------------
+sample output:
+
+![Annotation Output](acodet/src/imgs/annotation_output.png)
+
+Play around with the GUI in the __Online Demo__ here:
+https://acodet-web.streamlit.app/
+(the program will look identical when executed on your computer)
+
+__Video tutorials__ for installation and usage of the AcoDet GUI:
+https://www.youtube.com/watch?v=bJf4d8qf9h0&list=PLQOW4PvEYW-GNWIYRehs2-4-sa9T20c8A
+
+The corresponding paper to acodet can be found here: 
+https://doi.org/10.1121/10.0025275
+
+
+## Table of Contents
+- [Installation](#installation)
+    - [Installation on Windows](#installation-on-windows)
+    - [Installation on Mac](#installation-on-mac)
+    - [Installation on Linux](#installation-on-linux)
+- [Usage](#usage)
+    - [acodet Usage with GUI](#acodet-usage-with-gui)
+        - [Usecase 1: Generating annotations (GUI)](#usecase-1-generating-annotations-gui)
+        - [Usecase 2: Generating new training data (GUI)](#usecase-2-generating-new-training-data-gui)
+        - [Usecase 3: Training (GUI)](#usecase-3-training-gui)
+    - [acodet Usage headless](#acodet-usage-headless)
+        - [Usecase 1: Generating annotations](#usecase-1-generating-annotations)
+        - [Usecase 2: Generating new training data](#usecase-2-generating-new-training-data)
+        - [Usecase 3: Training](#usecase-3-training)
+- [Explanation of Sequence limit](#explanation-of-sequence-limit)
+- [Citation](#citation)
+- [FAQ](#faq)
+
+
+----------------------------------------------------
+# Installation
+## Installation on Windows
+### Preliminary software installations:
+- install python 3.8: (standard install, no admin privileges needed)
+<https://www.python.org/ftp/python/3.8.0/python-3.8.0-amd64.exe>
+- install git bash: (default install)
+<https://github.com/git-for-windows/git/releases/download/v2.38.1.windows.1/Git-2.38.1-64-bit.exe>
+
+### Installation instructions
+- create project directory in location of your choice
+- open git bash in project directory (right click, Git Bash here)
+- clone the repository:
+
+`git clone https://github.com/vskode/acodet.git`
+- Install virtualenv (copy and paste in Git Bash console):
+
+`"$HOME/AppData/Local/Programs/Python/Python38/python" -m pip install virtualenv`
+
+- Create a new virtual environment (default name env_acodet can be changed):
+
+ `"$HOME/AppData/Local/Programs/Python/Python38/python" -m virtualenv env_acodet`
+
+- activate newly created virtual environment (change env_acodet if necessary):
+
+`source env_acodet/Scripts/activate`
+
+- Install required packages:
+
+`pip install -r acodet/requirements.txt`
+
+-------------------------
+
+## Installation on Mac
+### Preliminary software installations:
+- install python 3.8: (standard install, no admin privileges needed)
+<https://www.python.org/ftp/python/3.8.7/python-3.8.7-macosx10.9.pkg>
+- install git: (default install)
+    - simply type `git` into the terminal and follow the installation instructions
+
+### Installation instructions
+- create project directory in location of your choice
+- open a terminal in the project directory
+- clone the repository:
+
+    `git clone https://github.com/vskode/acodet.git`
+- Install virtualenv (copy and paste in Git Bash console):
+
+    `/usr/bin/python/Python38/python -m pip install virtualenv`
+
+- Create a new virtual environment (default name env_acodet can be changed):
+
+    `/usr/bin/python/Python38/python -m virtualenv env_acodet`
+
+- activate newly created virtual environment (change env_acodet if necessary):
+
+    `source env_acodet/bin/activate`
+
+- Install required packages:
+    - if you have a M1 chip in your mac, run:
+
+        `pip install -r acodet/macM1_requirements/requirements_m1-1.txt`
+    - then run 
+
+        `pip install -r acodet/macM1_requirements/requirements_m1-1.txt`
+    
+    - if you have an older mac, run:
+
+        `pip install -r acodet/requirements.txt`
+
+--------------------------------------------
+## Installation on Linux
+### Preliminary software installations:
+- install python 3.8: (standard install, no admin privileges needed)
+<https://www.python.org/ftp/python/3.8.0/python-3.8.0-amd64.exe>
+- install git bash: (default install)
+<https://github.com/git-for-windows/git/releases/download/v2.38.1.windows.1/Git-2.38.1-64-bit.exe>
+
+### Installation instructions
+- create project directory in location of your choice
+- open a terminal in the project directory
+- clone the repository:
+
+    `git clone https://github.com/vskode/acodet.git`
+- Install virtualenv (copy and paste in Git Bash console):
+
+    `/usr/bin/python/Python38/python -m pip install virtualenv`
+
+- Create a new virtual environment (default name env_acodet can be changed):
+
+    `/usr/bin/python/Python38/python -m virtualenv env_acodet`
+
+- activate newly created virtual environment (change env_acodet if necessary):
+
+    `source env_acodet/bin/activate`
+
+- Install required packages:
+
+    `pip install -r acodet/requirements.txt`
+
+# Usage
+## AcoDet usage with GUI
+
+AcoDet provides a graphical user interface (GUI) for users to intuitively use the program. All inputs and outputs are handled through the GUI. To run the gui, run (while in acodet directory):
+
+`streamlit run streamlit_app.py`
+
+This should start a new tab in a web browser which runs the interface that you can interact with. It is important that your virtual environment where you have installed the required packages is active, for that see the Installation sections. To activate the environment run 
+
+`source ../env_acodet/Scripts/activate` (on Windows) 
+or 
+
+`source ../env_acodet/bin/activate` (on Mac/Linux) 
+
+while your terminal directory is inside **acodet**. 
+
+### Usecase 1: generating annotations (GUI)
+
+- Choose the 1 - Inference option from the first drop-down menu
+- Choose between the predefined Settings
+0. run all of the steps
+1. generating new annotations 
+2. filterin existing annotations
+3. generating hourly predictions
+- click Next
+- Depending on your choice you will be prompted to enter the path leading to either your sound files our existing annotation files 
+    - Enter a path in the text field that is one directory above the folder you would like to use
+    - In the dropdown menu you will be presented with all the folders inside the specified path. Choose the one you would like to work with
+    - **Important**: time stamps are required within the file names of the source files for steps 0. and 3.
+- If required, choose a Model threshold
+- click Run computation
+- A progress bar should show the progress of your computations
+- click Show results
+- The Output section will provide you with information of the location of the files, and depending on your choice of predifines Settings will show different tabs.
+    - the "Stats" is an overview of all processed files, with timestamp and number of predictions
+    - the "Annot. Files" gives you a dropdown menu where you can look into prediction values for each vocalization within each source file. By default the threshold for this will be at 0.5, meaning that all sections with prediction values below that will be discarded.
+    - the "Filtered Files" shows the same as the previous tab, however, it only shows sections with previously defined values exceeding the predefined threshold.
+    - the "Annotaion Plots" shows you a visualization revealing the number of anotations per hour in your dataset. Choose your dataset from the dropdown (in some cases there is only one dataset inside your previously defined folder). 
+        - The calculations behind this visualization is explained in detail in the corresponding journal paper that is currenlty under review and will be linked here as soon as published.
+        - You can choose between a "Simple limit" and a "Sequence limit" 
+            - the main distinction is whether consecute vocalizations are required for them to be counted (this should help reduce false positives)
+            - the "Simple limit" will compute much faster than the "Sequence limit"
+        - You can also change the threshold of the model predictions which will then allo you to update the visualization. If the "Sequence limit" is chose, the number limit can also be changed, which will change the required number of consecutive vocalizations for them to be counted. (Try it out)
+        - All visualizations can be exported as .png files by clicking on the small camera icon in the top right.
+    - the "Presence Plots" shows a similar visualization as the previous section, however, only showing binary presence.
+
+
+### Usecase 2: generating new training data (GUI)
+
+This feature is currently not integrated in the gui.
+### Usecase 3: training (GUI)
+
+This feature is currently not integrated in the gui.
+
+## AcoDet usage headless
+Users only need to change the files **simple_congif.yml** and **advanced_config.yml** to use AcoDet. Once the config files are changed, users can run the program by running the command `python run.py` inside the **acodet** directory.
+
+### Usecase 1: generating annotations
+To generate annotations:
+- open the file **simple_config.yml** in any Editor (default is Notepad). 
+- change `run_config` to `1`
+- change `predefined_settings` to one of the following:
+    - `1` for generating annotations with a threshold of 0.5
+    - `2` for generating annotations with a custom threshold
+        - specify threshold (**thresh**) value in **simple_config.yml** (defaults to 0.9)
+    - `3` for generating hourly counts and presence spreadsheets and visualizations (using the sequence criterion and the simple limit)
+        - _simple limit_ and _sequence criterion_ are accumulation metrics aiming to deliver hourly presence information, while filtering out false positives
+            - _simple limit_ -> only consider annotations if the number of annotations exceeding the **thresh** value is higher than the value for **simple_limit** in **simple_config.yml** (in a given hour in the dataset)
+            - _sequence criterion_ -> only consider annotations if the number of consecutive annotations within **sc_con_win** number of windows exceeding the **sc_thresh** value is higher than **sc_limit** (in a given hour in the dataset)
+        - hourly counts gives the number of annotations according to the accumulation metrics
+        - hourly presence gives a binary (0 -> no whale; 1 -> whale) corresponding to whether the accumulation metrics are satisfied
+    - `4` for generating hourly counts and presence spreadsheets and visualizations (using only the simple limit)
+    - or `0` to run all of the above in sequece
+- change `sound_files_source` to the top level directory containing the dataset(s) you want to annotate
+
+- once finished, save the **simple_config.yml** file
+
+To start the program:
+- activate the virtual environment again:
+
+`source env_acodet/Scripts/activate`
+
+- run the run.py script:
+
+`python acodet/run.py`
+
+### Output
+
+The software will now run thorugh your dataset and gerate annotations for every (readable) soundifle within the dataset. While running, a spreadsheet, called stats.csv is continuously updated showing information on the annotations for every file (do not open while program is still running, because the program wont be able to access it).
+
+The program will create a directory called `generated_annotatoins` in the project directory. It will then create a directory corresponding to the date and time that you started the annotation process. Within that directory you will find a directory `thresh_0.5` corresponding to all annotations with a threshold of 0.5. Furthermore you will find the `stats.csv` spreadsheet.
+
+If you have chosen option 2 (or 0) you will also find a directory `thresh_0.x` where the x stands for the custom threshold you specified in the **simple_config.yml** file. Within the `thresh` directories you will find the name of your dataset. 
+
+If you have chosen option 3, 4 or 0 you will find a directory `analysis` within the dataset directory. In that directory you will find spreadsheets for hourly presence and hourly counts, as well as visualizations of the hourly presence and hourly counts.
+
+### Usecase 2: generating new training data
+
+Either use manually created annotations -> option 2, or create new annotations by reviewing the automatically generated annotations -> option 1.
+
+For option 1, use Raven to open sound files alongside their automatically generated annotations. Edit the column `Predictions/Comments` by writing `n` for noise, `c` for call, or `u` for undefined. If the majority of the shown windows are calls, add the suffix `_allcalls` before the `.txt` ending so that the program will automatically label all of the windows as calls, unless specified as `n`, `c`, or `u`. The suffix `_allnoise` will do the same for noise. The suffix `_annotated` will label all unchanged windows as undefined - thereby essentially ignoring them for the created dataset.
+
+Once finished, insert the top-level directory path to the `reviewed_annotation_source` variable in **simple_config.yml**. 
+
+To generate new training data:
+- open the file **simple_config.yml** in any Editor (default is Notepad). 
+- change `run_config` to `2`
+- change `predefined_settings` to one of the following:
+    - `1` for generating training data from reviewed annotations
+    - `2` for generating training data from manually created training data (space in between annotations will be interpretted as noise)
+- change `sound_files_source` to the top level directory containing the dataset(s) containing the sound files
+
+- once finished, save the **simple_config.yml** file
+
+To start the program:
+- activate the virtual environment again:
+
+`source env_acodet/Scripts/activate`
+
+- run the run.py script:
+
+`python acodet/run.py`
+
+### Usecase 3: training
+
+To train the model:
+- open the file **simple_config.yml** in any Editor (default is Notepad). 
+- change `run_config` to `3`
+- change `predefined_settings` to one of the following:
+    - `1` for generating training data from reviewed annotations
+
+- once finished, save the **simple_config.yml** file
+- more adcanced changes for model parameters can be done in **advanced_config.yml**
+
+To start the program:
+- activate the virtual environment again:
+
+`source env_acodet/Scripts/activate`
+
+- run the run.py script:
+
+`python acodet/run.py`
+
+# Explanation of Sequence limit
+Besides a simple thresholding (simple limit) the sequence limit can be used to distinguish repeating vocalizations from other noise sources. For humpback whales this vastly reduces the number of generated false positives.
+
+To briefly explain:
+In a stream of 20 consecutive windows (of approx. 4 s length) you can set limit and threshold. After applying the limit, predictions are only kept if they exceed the threshold and occur in thre frequency set by the limit. This is especially convenient for __hourly presence__ annotations. The below image shows an example of 20 consecutive windows with their given model prediction values. The highlighted values exceed the threshold and since they occur in the required frequency (Limit=3), the hourly presence yield the value 1.
+
+![Sequence Limit](acodet/src/imgs/sequence_limit.png)
+
+The sequence limit can be useful in noisy environments, where vocalizations are masked by noise and their repetitiveness can be used to distinguish them from irregular background noise.
+
+Threshold and limit can be set interactively and their effect on the data can be analyzed right away. As this is only post-processing the existing annotations, computing time is very fast. The following image shows a screenshot of the sequence limit in the acodet GUI.
+
+![alt text](acodet/src/imgs/gui_sequence_limit.png)
+
+# Citation
+
+If you used acodet in your work, please reference the following:
+
+Vincent Kather, Fabian Seipel, Benoit Berges, Genevieve Davis, Catherine Gibson, Matt Harvey, Lea-Anne Henry, Andrew Stevenson, Denise Risch; Development of a machine learning detector for North Atlantic humpback whale song. J. Acoust. Soc. Am. 1 March 2024; 155 (3): 2050–2064.
+
+For bibtex:
+```bibtex
+@article{10.1121/10.0025275,
+    author = {Kather, Vincent and Seipel, Fabian and Berges, Benoit and Davis, Genevieve and Gibson, Catherine and Harvey, Matt and Henry, Lea-Anne and Stevenson, Andrew and Risch, Denise},
+    title = "{Development of a machine learning detector for North Atlantic humpback whale song}",
+    journal = {The Journal of the Acoustical Society of America},
+    volume = {155},
+    number = {3},
+    pages = {2050-2064},
+    year = {2024},
+    month = {03},
+    issn = {0001-4966},
+    doi = {10.1121/10.0025275}
+}
+```
+
+# FAQ
+
+At the moment the generation of new training data and the training are not yet supported in the graphical user interface.

acodet/annotate.py

@@ -0,0 +1,229 @@
+import time
+from acodet import models
+from acodet.funcs import (
+    get_files,
+    gen_annotations,
+    get_dt_filename,
+    remove_str_flags_from_predictions,
+)
+from acodet import global_config as conf
+import pandas as pd
+import numpy as np
+from pathlib import Path
+
+
+class MetaData:
+    def __init__(self):
+        """
+        Initialize the MetaData class with the columns that will be used to
+        store the metadata of the generated annotations.
+        """
+        self.filename = "filename"
+        self.f_dt = "date from timestamp"
+        self.n_pred_col = "number of predictions"
+        self.avg_pred_col = "average prediction value"
+        self.n_pred08_col = "number of predictions with thresh>0.8"
+        self.n_pred09_col = "number of predictions with thresh>0.9"
+        self.time_per_file = "computing time [s]"
+        if not "timestamp_folder" in conf.session:
+            self.df = pd.DataFrame(
+                columns=[
+                    self.filename,
+                    self.f_dt,
+                    self.n_pred_col,
+                    self.avg_pred_col,
+                    self.n_pred08_col,
+                    self.n_pred09_col,
+                ]
+            )
+        else:
+            self.df = pd.read_csv(
+                conf.session["timestamp_folder"].parent.parent.joinpath(
+                    "stats.csv"
+                )
+            )
+            self.df.pop("Unnamed: 0")
+
+    def append_and_save_meta_file(
+        self,
+        file: Path,
+        annot: pd.DataFrame,
+        f_ind: int,
+        timestamp_foldername: str,
+        relativ_path: str = conf.SOUND_FILES_SOURCE,
+        computing_time: str = "not calculated",
+        **kwargs,
+    ):
+        """
+        Append the metadata of the generated annotations to the dataframe and
+        save it to a csv file.
+
+        Parameters
+        ----------
+        file : Path
+            Path to the file that was annotated.
+        annot : pd.DataFrame
+            Dataframe containing the annotations.
+        f_ind : int
+            Index of the file.
+        timestamp_foldername : str
+            Timestamp of the annotation run for folder name.
+        relativ_path : str, optional
+            Path of folder containing files , by default conf.SOUND_FILES_SOURCE
+        computing_time : str, optional
+            Amount of time that prediction took, by default "not calculated"
+        """
+        self.df.loc[f_ind, self.f_dt] = str(get_dt_filename(file).date())
+        self.df.loc[f_ind, self.filename] = Path(file).relative_to(
+            relativ_path
+        )
+        # TODO relative_path muss noch dauerhaft geändert werden
+        self.df.loc[f_ind, self.n_pred_col] = len(annot)
+        df_clean = remove_str_flags_from_predictions(annot)
+        self.df.loc[f_ind, self.avg_pred_col] = np.mean(
+            df_clean[conf.ANNOTATION_COLUMN]
+        )
+        self.df.loc[f_ind, self.n_pred08_col] = len(
+            df_clean.loc[df_clean[conf.ANNOTATION_COLUMN] > 0.8]
+        )
+        self.df.loc[f_ind, self.n_pred09_col] = len(
+            df_clean.loc[df_clean[conf.ANNOTATION_COLUMN] > 0.9]
+        )
+        self.df.loc[f_ind, self.time_per_file] = computing_time
+        self.df.to_csv(
+            Path(conf.GEN_ANNOTS_DIR)
+            .joinpath(timestamp_foldername)
+            .joinpath("stats.csv")
+        )
+
+
+def run_annotation(train_date=None, **kwargs):
+    files = get_files(location=conf.SOUND_FILES_SOURCE, search_str="**/*")
+    if not "timestamp_folder" in conf.session:
+        timestamp_foldername = time.strftime(
+            "%Y-%m-%d_%H-%M-%S", time.gmtime()
+        )
+        timestamp_foldername += conf.ANNOTS_TIMESTAMP_FOLDER
+        mdf = MetaData()
+        f_ind = 0
+
+    else:
+        timestamp_foldername = conf.session[
+            "timestamp_folder"
+        ].parent.parent.stem
+        last_annotated_file = list(
+            conf.session["timestamp_folder"].rglob("*.txt")
+        )[-1]
+        file_stems = [f.stem for f in files]
+        file_idx = np.where(
+            np.array(file_stems) == last_annotated_file.stem.split("_annot")[0]
+        )[0][0]
+        files = files[file_idx:]
+        mdf = MetaData()
+        f_ind = file_idx - 1
+
+    if not train_date:
+        model = models.init_model()
+        mod_label = conf.MODEL_NAME
+    else:
+        df = pd.read_csv("../trainings/20221124_meta_trainings.csv")
+        row = df.loc[df["training_date"] == train_date]
+        model_name = row.Model.values[0]
+        keras_mod_name = row.keras_mod_name.values[0]
+
+        model = models.init_model(
+            model_instance=model_name,
+            checkpoint_dir=f"../trainings/{train_date}/unfreeze_no-TF",
+            keras_mod_name=keras_mod_name,
+        )
+        mod_label = train_date
+
+    if conf.STREAMLIT:
+        import streamlit as st
+
+        st.session_state.progbar1 = 0
+    for i, file in enumerate(files):
+        if file.is_dir():
+            continue
+
+        if conf.STREAMLIT:
+            import streamlit as st
+
+            st.session_state.progbar1 += 1
+        f_ind += 1
+        start = time.time()
+        annot = gen_annotations(
+            file,
+            model,
+            mod_label=mod_label,
+            timestamp_foldername=timestamp_foldername,
+            num_of_files=len(files),
+            **kwargs,
+        )
+        computing_time = time.time() - start
+        mdf.append_and_save_meta_file(
+            file,
+            annot,
+            f_ind,
+            timestamp_foldername,
+            computing_time=computing_time,
+            **kwargs,
+        )
+    return timestamp_foldername
+
+
+def check_for_multiple_time_dirs_error(path):
+    if not path.joinpath(conf.THRESH_LABEL).exists():
+        subdirs = [l for l in path.iterdir() if l.is_dir()]
+        path = path.joinpath(subdirs[-1].stem)
+    return path
+
+
+def filter_annots_by_thresh(time_dir=None, **kwargs):
+    if not time_dir:
+        path = Path(conf.GEN_ANNOT_SRC)
+    else:
+        path = Path(conf.GEN_ANNOTS_DIR).joinpath(time_dir)
+    files = get_files(location=path, search_str="**/*txt")
+    files = [f for f in files if conf.THRESH_LABEL in str(f.parent)]
+    path = check_for_multiple_time_dirs_error(path)
+    for i, file in enumerate(files):
+        try:
+            annot = pd.read_csv(file, sep="\t")
+        except Exception as e:
+            print(
+                "Could not process file, maybe not an annotation file?",
+                "Error: ",
+                e,
+            )
+        annot = annot.loc[annot[conf.ANNOTATION_COLUMN] >= conf.THRESH]
+        save_dir = (
+            path.joinpath(f"thresh_{conf.THRESH}")
+            .joinpath(file.relative_to(path.joinpath(conf.THRESH_LABEL)))
+            .parent
+        )
+        save_dir.mkdir(exist_ok=True, parents=True)
+
+        if "Selection" in annot.columns:
+            annot.index.name = "Selection"
+            annot = annot.drop(columns=["Selection"])
+        else:
+            annot.index = np.arange(1, len(annot) + 1)
+            annot.index.name = "Selection"
+        annot.to_csv(save_dir.joinpath(file.stem + file.suffix), sep="\t")
+        if conf.STREAMLIT and "progbar1" in kwargs.keys():
+            kwargs["progbar1"].progress((i + 1) / len(files), text="Progress")
+        else:
+            print(f"Writing file {i+1}/{len(files)}")
+    if conf.STREAMLIT:
+        return path
+
+
+if __name__ == "__main__":
+    train_dates = ["2022-11-30_01"]
+
+    for train_date in train_dates:
+        start = time.time()
+        run_annotation(train_date)
+        end = time.time()
+        print(end - start)

acodet/annotation_mappers.json

@@ -0,0 +1,11 @@
+{
+    "default_mapper": {
+        "Begin Time (s)": "start", 
+        "End Time (s)": "end",
+        "Low Freq (Hz)" : "freq_min", 
+        "High Freq (Hz)" : "freq_max"},
+    "file_offset_mapper": {
+        "File Offset (s)": "start", 
+        "Low Freq (Hz)" : "freq_min", 
+        "High Freq (Hz)" : "freq_max"} 
+}

acodet/augmentation.py

@@ -0,0 +1,185 @@
+import tensorflow as tf
+from keras_cv.layers import BaseImageAugmentationLayer
+from acodet.plot_utils import plot_sample_spectrograms
+import tensorflow_io as tfio
+
+AUTOTUNE = tf.data.AUTOTUNE
+
+
+class CropAndFill(BaseImageAugmentationLayer):
+    def __init__(self, height: int, width: int, seed: int = None) -> None:
+        """
+        Augmentation class inheriting from keras' Augmentation Base class.
+        This class takes images, cuts them at a random x position and then
+        appends the first section to the second section. It is intended for
+        spectrograms of labelled bioacoustics data. This way the vocalization
+        in the spectrogram is time shifted and potentially cut. All of which
+        is possible to occur due to a windowing of a recording file that is
+        intended for inference.
+        It is essentially a time shift augmentation whilst preserving window
+        length and not requiring reloading data from the source file.
+
+        Args:
+            height (int): height of image
+            width (int): width of image
+            seed (int, optional): create randomization seed. Defaults to None.
+        """
+        super().__init__()
+        self.height = height
+        self.width = width
+        self.seed = seed
+        tf.random.set_seed(self.seed)
+
+    def call(self, audio: tf.Tensor):
+        """
+        Compute time shift augmentation by creating a random slicing
+        position and then returning the reordered image.
+
+        Args:
+            audio (tf.Tensor): input image
+
+        Returns:
+            tf.Tensor: reordered image
+        """
+        beg = tf.random.uniform(
+            shape=[], maxval=self.width // 2, dtype=tf.int32
+        )
+
+        # for debugging purposes
+        # tf.print('time shift augmentation computed)
+
+        return tf.roll(audio, shift=[beg], axis=[0])
+
+
+def time_shift():
+    return tf.keras.Sequential([CropAndFill(64, 128)])
+
+
+def m_test(ds1, ds2, alpha=0.4):
+    call, lab = ds1
+    noise, l = ds2
+    noise_alpha = alpha * tf.math.reduce_max(noise)
+    train_alpha = (1 - alpha) * tf.math.reduce_max(call)
+    # tf.print('performing mixup')
+    return (call * train_alpha + noise * noise_alpha, lab)
+
+
+def time_mask(x, y, spec_param=10):
+    # tf.print('performing time_mask')
+    return (tfio.audio.time_mask(x, param=spec_param), y)
+
+
+def freq_mask(x, y, spec_param=10):
+    # tf.print('performing freq_mask')
+    return (tfio.audio.freq_mask(x, param=spec_param), y)
+
+
+##############################################################################
+##############################################################################
+##############################################################################
+
+
+def run_augment_pipeline(
+    ds,
+    noise_data,
+    noise_set_size,
+    train_set_size,
+    time_augs,
+    mixup_augs,
+    seed=None,
+    plot=False,
+    time_start=None,
+    spec_aug=False,
+    spec_param=10,
+    **kwargs,
+):
+    T = time_shift()
+    if plot:
+        plot_sample_spectrograms(
+            ds,
+            dir=time_start,
+            name="train",
+            seed=seed,
+            ds_size=train_set_size,
+            **kwargs,
+        )
+
+    if mixup_augs:
+        ds_n = noise_data.repeat(train_set_size // noise_set_size + 1)
+        if plot:
+            plot_sample_spectrograms(
+                ds_n,
+                dir=time_start,
+                name=f"noise",
+                seed=seed,
+                ds_size=train_set_size,
+                **kwargs,
+            )
+
+        if plot:
+            dss = tf.data.Dataset.zip((ds, ds_n))
+            ds_mu = dss.map(
+                lambda x, y: m_test(x, y), num_parallel_calls=AUTOTUNE
+            )
+            plot_sample_spectrograms(
+                ds_mu,
+                dir=time_start,
+                name=f"augment_0-MixUp",
+                seed=seed,
+                ds_size=train_set_size,
+                **kwargs,
+            )
+
+        ds_n = ds_n.shuffle(train_set_size // noise_set_size + 1)
+        dss = tf.data.Dataset.zip((ds, ds_n))
+        ds_mu = dss.map(lambda x, y: m_test(x, y), num_parallel_calls=AUTOTUNE)
+        ds_mu_n = ds_mu.concatenate(noise_data)
+        # ds = ds.concatenate(ds_mu_n)
+
+    if time_augs:
+        ds_t = ds.map(
+            lambda x, y: (T(x, training=True), y), num_parallel_calls=AUTOTUNE
+        )
+        if plot:
+            plot_sample_spectrograms(
+                ds_t,
+                dir=time_start,
+                name=f"augment_0-TimeShift",
+                seed=seed,
+                ds_size=train_set_size,
+                **kwargs,
+            )
+        # ds = ds.concatenate(ds_t)
+
+    if spec_aug:
+        ds_tm = ds.map(time_mask)
+        # ds = ds.concatenate(ds_tm)
+        if plot:
+            plot_sample_spectrograms(
+                ds_tm,
+                dir=time_start,
+                name=f"augment_0-TimeMask",
+                seed=seed,
+                ds_size=train_set_size,
+                **kwargs,
+            )
+        ds_fm = ds.map(freq_mask)
+        # ds = ds.concatenate(ds_fm)
+        if plot:
+            plot_sample_spectrograms(
+                ds_fm,
+                dir=time_start,
+                name=f"augment_0-TFreqMask",
+                seed=seed,
+                ds_size=train_set_size,
+                **kwargs,
+            )
+    if mixup_augs:
+        ds = ds.concatenate(ds_mu_n)
+    if time_augs:
+        ds = ds.concatenate(ds_t)
+    if spec_aug:
+        ds = ds.concatenate(ds_tm)
+        ds = ds.concatenate(ds_fm)
+
+    return ds

acodet/combine_annotations.py

@@ -0,0 +1,293 @@
+import pandas as pd
+
+pd.options.mode.chained_assignment = None  # default='warn'
+import glob
+from pathlib import Path
+from acodet.funcs import remove_str_flags_from_predictions, get_files
+import os
+import numpy as np
+import acodet.global_config as conf
+import json
+
+with open("acodet/annotation_mappers.json", "r") as m:
+    mappers = json.load(m)
+
+
+# TODO aufraeumen
+# annotation_files = Path(r'/mnt/f/Daten/20221019-Benoit/').glob('**/*.txt')
+# annotation_files = Path(r'generated_annotations/2022-11-04_12/').glob('ch*.txt')
+
+
+def compensate_for_naming_inconsistencies(hard_drive_path, file):
+    split_file = file.stem.split("Table")[0]
+    file_path = glob.glob(
+        f"{hard_drive_path}/**/{split_file}*wav", recursive=True
+    )
+
+    if not file_path:
+        file_tolsta = "336097327." + split_file[6:].replace(
+            "_000", ""
+        ).replace("_", "")
+        file_path = glob.glob(
+            f"{hard_drive_path}/**/{file_tolsta}*wav", recursive=True
+        )
+
+    if not file_path:
+        file_tolsta = "335564853." + split_file[6:].replace(
+            "5_000", "4"
+        ).replace("_", "")
+        file_path = glob.glob(
+            f"{hard_drive_path}/**/{file_tolsta}*wav", recursive=True
+        )
+
+    if not file_path:
+        file_new_annot = file.stem.split("_annot")[0]
+        file_path = glob.glob(
+            f"{hard_drive_path}/**/{file_new_annot}*", recursive=True
+        )
+
+    if not file_path:
+        split_file_underscore = file.stem.split("_")[0]
+        file_path = glob.glob(
+            f"{hard_drive_path}/**/{file_new_annot}*wav", recursive=True
+        )
+        if not file_path:
+            file_new_annot = split_file_underscore.split(".")[-1]
+            file_path = glob.glob(
+                f"{hard_drive_path}/**/*{file_new_annot}*wav", recursive=True
+            )
+
+    if not file_path:
+        print("sound file could not be found, continuing with next file")
+        return False
+    return file_path
+
+
+def get_corresponding_sound_file(file):
+    hard_drive_path = conf.SOUND_FILES_SOURCE
+    file_path = glob.glob(
+        f"{hard_drive_path}/**/{file.stem}*wav", recursive=True
+    )
+    if not file_path:
+        file_path = compensate_for_naming_inconsistencies(
+            hard_drive_path, file
+        )
+
+    if not file_path:
+        # TODO fehler raisen, dass bringt so einfach nichts
+        return "empty"
+
+    if len(file_path) > 1:
+        p_dir = list(file.relative_to(conf.REV_ANNOT_SRC).parents)[-2]
+        p_dir_main = str(p_dir).split("_")[0]
+        for path in file_path:
+            if p_dir_main in path:
+                file_path = path
+    else:
+        file_path = file_path[0]
+
+    if isinstance(file_path, list) and len(file_path) > 1:
+        file_path = file_path[0]
+        print(
+            "WARNING: Multiple sound files for annotations file found."
+            " Because pattern could not be resolved, first file is chosen."
+            f"\nannotations file name: \n{file}\n"
+            f"sound file name: \n{file_path}\n"
+        )
+
+    return file_path
+
+
+def seperate_long_annotations(df):
+    bool_long_annot = df["End Time (s)"] - df["Begin Time (s)"] > round(
+        conf.CONTEXT_WIN / conf.SR
+    )
+    for i, row in df.loc[bool_long_annot].iterrows():
+        n_new_annots = int(
+            (row["End Time (s)"] - row["Begin Time (s)"])
+            / (conf.CONTEXT_WIN / conf.SR)
+        )
+        begins = row["Begin Time (s)"] + np.arange(n_new_annots) * (
+            conf.CONTEXT_WIN / conf.SR
+        )
+        ends = begins + (conf.CONTEXT_WIN / conf.SR)
+        n_df = pd.DataFrame()
+        for col in row.keys():
+            n_df[col] = [row[col]] * n_new_annots
+        n_df["Begin Time (s)"] = begins
+        n_df["End Time (s)"] = ends
+        n_df["Selection"] = np.arange(n_new_annots) + row["Selection"]
+        df = pd.concat(
+            [df.drop(row.name), n_df]
+        )  # delete long annotation from df
+    return df
+
+
+def label_explicit_noise(df):
+    df_clean = remove_str_flags_from_predictions(df)
+
+    expl_noise_crit_idx = np.where(df_clean[conf.ANNOTATION_COLUMN] > 0.9)[0]
+    df.loc[expl_noise_crit_idx, "label"] = "explicit 0"
+    return df
+
+
+def differentiate_label_flags(df, flag=None):
+    df.loc[:, conf.ANNOTATION_COLUMN].fillna(value="c", inplace=True)
+    df.loc[df[conf.ANNOTATION_COLUMN] == "c", "label"] = 1
+    df.loc[df[conf.ANNOTATION_COLUMN] == "n", "label"] = "explicit 0"
+    df_std = seperate_long_annotations(df)
+
+    df_std = df_std.drop(
+        df_std.loc[df_std[conf.ANNOTATION_COLUMN] == "u"].index
+    )
+    df_std.index = pd.RangeIndex(0, len(df_std))
+    if flag == "noise":
+        df_std = label_explicit_noise(df_std)
+
+    return df_std
+
+
+def get_labels(file, df, active_learning=False, **kwargs):
+    if not active_learning:
+        df["label"] = 1
+    else:
+        noise_flag, annotated_flag, calls_flag = [
+            "_allnoise",
+            "_annotated",
+            "_allcalls",
+        ]
+        df = df.iloc[df.Selection.drop_duplicates().index]
+        if calls_flag in file.stem:
+            df["label"] = 1
+            df = differentiate_label_flags(df, flag="calls")
+        elif noise_flag in file.stem:
+            df["label"] = 0
+            df = differentiate_label_flags(df, flag="noise")
+        elif annotated_flag in file.stem:
+            df_clean = remove_str_flags_from_predictions(df)
+            df.loc[df_clean.index, conf.ANNOTATION_COLUMN] = "u"
+            df = differentiate_label_flags(df)
+    return df
+
+
+def standardize(
+    df, *, mapper, filename_col="filename", selection_col="Selection"
+):
+    keep_cols = ["label", "start", "end", "freq_min", "freq_max"]
+    df = df.rename(columns=mapper)
+    if not "end" in df.columns:
+        df["end"] = df.start + (df["End Time (s)"] - df["Begin Time (s)"])
+    out_df = df[keep_cols]
+    out_df.index = pd.MultiIndex.from_arrays(
+        arrays=(df[filename_col], df[selection_col])
+    )
+    return out_df.astype(dtype=np.float64)
+
+
+def filter_out_high_freq_and_high_transient(df):
+    df = df.loc[df["High Freq (Hz)"] <= 2000]
+    df = df.loc[df["End Time (s)"] - df["Begin Time (s)"] >= 0.4]
+    return df
+
+
+def finalize_annotation(file, freq_time_crit=False, **kwargs):
+    ann = pd.read_csv(file, sep="\t")
+
+    ann["filename"] = get_corresponding_sound_file(file)
+    # if not ann['filename']:
+    #     print(f'corresponding sound file for annotations file: {file} not found')
+
+    ann = get_labels(file, ann, **kwargs)
+    if "File Offset (s)" in ann.columns:
+        mapper = mappers["file_offset_mapper"]
+    else:
+        mapper = mappers["default_mapper"]
+
+    if freq_time_crit:
+        ann = filter_out_high_freq_and_high_transient(ann)
+
+    ann_explicit_noise = ann.loc[ann["label"] == "explicit 0", :]
+    ann_explicit_noise["label"] = 0
+    ann = ann.drop(ann.loc[ann["label"] == "explicit 0"].index)
+    std_annot_train = standardize(ann, mapper=mapper)
+    std_annot_enoise = standardize(ann_explicit_noise, mapper=mapper)
+
+    return std_annot_train, std_annot_enoise
+
+
+def leading_underscore_in_parent_dirs(file):
+    return "_" in [f.stem[0] for f in list(file.parents)[:-1]]
+
+
+def get_active_learning_files(files):
+    cases = ["_allnoise", "_annotated", "_allcalls"]
+    cleaned_files = [f for f in files if [True for c in cases if c in f.stem]]
+    drop_cases = ["_tobechecked"]
+    final_cleanup = [
+        f
+        for f in cleaned_files
+        if not [True for d in drop_cases if d in f.stem]
+    ]
+    return final_cleanup
+
+
+def generate_final_annotations(
+    annotation_files=None, active_learning=True, **kwargs
+):
+    if not annotation_files:
+        annotation_files = get_files(
+            location=conf.REV_ANNOT_SRC, search_str="**/*.txt"
+        )
+    files = list(annotation_files)
+    if active_learning:
+        files = get_active_learning_files(files)
+    folders, counts = np.unique(
+        [list(f.relative_to(conf.REV_ANNOT_SRC).parents) for f in files],
+        return_counts=True,
+    )
+    if len(folders) > 1:
+        folders, counts = np.unique(
+            [
+                list(f.relative_to(conf.REV_ANNOT_SRC).parents)[-2]
+                for f in files
+            ],
+            return_counts=True,
+        )
+    files.sort()
+    ind = 0
+    for i, folder in enumerate(folders):
+        df_t, df_n = pd.DataFrame(), pd.DataFrame()
+        for _ in range(counts[i]):
+            if leading_underscore_in_parent_dirs(files[ind]):
+                print(
+                    files[ind],
+                    " skipped due to leading underscore in parent dir.",
+                )
+                ind += 1
+                continue
+            df_train, df_enoise = finalize_annotation(
+                files[ind],
+                all_noise=False,
+                active_learning=active_learning,
+                **kwargs,
+            )
+            df_t = pd.concat([df_t, df_train])
+            df_n = pd.concat([df_n, df_enoise])
+            print(f"Completed file {ind}/{len(files)}.", end="\r")
+            ind += 1
+
+        # TODO include date in path by default
+        save_dir = Path(conf.ANNOT_DEST).joinpath(folder)
+        save_dir.mkdir(exist_ok=True, parents=True)
+        df_t.to_csv(save_dir.joinpath("combined_annotations.csv"))
+        df_n.to_csv(save_dir.joinpath("explicit_noise.csv"))
+    # save_ket_annot_only_existing_paths(df)
+
+
+if __name__ == "__main__":
+    annotation_files = list(Path(conf.REV_ANNOT_SRC).glob("**/*.txt"))
+    if len(annotation_files) == 0:
+        annotation_files = list(Path(conf.REV_ANNOT_SRC).glob("*.txt"))
+    generate_final_annotations(
+        annotation_files, active_learning=True, freq_time_crit=False
+    )

acodet/create_session_file.py

@@ -0,0 +1,29 @@
+import yaml
+import json
+import streamlit as st
+
+
+def create_session_file():
+    with open("simple_config.yml", "r") as f:
+        simple = yaml.safe_load(f)
+
+    with open("advanced_config.yml", "r") as f:
+        advanced = yaml.safe_load(f)
+
+    session = {**simple, **advanced}
+
+    if "session_started" in st.session_state:
+        for k, v in session.items():
+            setattr(st.session_state, k, v)
+    else:
+        with open("acodet/src/tmp_session.json", "w") as f:
+            json.dump(session, f)
+
+
+def read_session_file():
+    if "session_started" in st.session_state:
+        session = {**st.session_state}
+    else:
+        with open("acodet/src/tmp_session.json", "r") as f:
+            session = json.load(f)
+    return session

acodet/evaluate.py

@@ -0,0 +1,253 @@
+from acodet.plot_utils import (
+    plot_evaluation_metric,
+    plot_model_results,
+    plot_sample_spectrograms,
+    plot_pr_curve,
+)
+from acodet import models
+from acodet.models import get_labels_and_preds
+from acodet.tfrec import run_data_pipeline, spec
+import matplotlib.pyplot as plt
+import matplotlib.colors as mcolors
+from pathlib import Path
+import pandas as pd
+import time
+import numpy as np
+from acodet.humpback_model_dir import front_end
+import acodet.global_config as conf
+
+
+def get_info(date):
+    keys = [
+        "data_path",
+        "batch_size",
+        "epochs",
+        "Model",
+        "keras_mod_name",
+        "load_weights",
+        "training_date",
+        "steps_per_epoch",
+        "f_score_beta",
+        "f_score_thresh",
+        "bool_SpecAug",
+        "bool_time_shift",
+        "bool_MixUps",
+        "weight_clipping",
+        "init_lr",
+        "final_lr",
+        "unfreezes",
+        "preproc blocks",
+    ]
+    path = Path(f"../trainings/{date}")
+    f = pd.read_csv(path.joinpath("training_info.txt"), sep="\t")
+    l, found = [], 0
+    for key in keys:
+        found = 0
+        for s in f.values:
+            if key in s[0]:
+                l.append(s[0])
+                found = 1
+        if found == 0:
+            l.append(f"{key}= nan")
+    return {key: s.split("= ")[-1] for s, key in zip(l, keys)}
+
+
+def write_trainings_csv():
+    trains = list(Path("../trainings").iterdir())
+    try:
+        df = pd.read_csv("../trainings/20221124_meta_trainings.csv")
+        new = [t for t in trains if t.stem not in df["training_date"].values]
+        i = len(trains) - len(new) + 1
+        trains = new
+    except Exception as e:
+        print("file not found", e)
+        df = pd.DataFrame()
+        i = 0
+    for path in trains:
+        try:
+            f = pd.read_csv(path.joinpath("training_info.txt"), sep="\t")
+            for s in f.values:
+                if "=" in s[0]:
+                    df.loc[i, s[0].split("= ")[0].replace(" ", "")] = s[
+                        0
+                    ].split("= ")[-1]
+                    df.loc[i, "training_date"] = path.stem
+            i += 1
+        except Exception as e:
+            print(e)
+        df.to_csv("../trainings/20230207_meta_trainings.csv")
+
+
+def create_overview_plot(
+    train_dates=[],
+    val_set=None,
+    display_keys=["Model"],
+    plot_metrics=False,
+    titles=None,
+):
+    if not train_dates:
+        train_dates = "2022-11-30_01"
+    if not isinstance(train_dates, list):
+        train_dates = [train_dates]
+
+    df = pd.read_csv("../trainings/20230207_meta_trainings.csv")
+    df.index = df["training_date"]
+
+    if not val_set:
+        val_set = list(Path(conf.TFREC_DESTINATION).iterdir())
+        if "dataset_meta_train" in [f.stem for f in val_set]:
+            val_set = val_set[0].parent
+
+    if isinstance(val_set, list):
+        val_label = "all"
+    else:
+        val_label = Path(val_set).stem
+
+    string = str("Model:{}; " f"val: {val_label}")
+    if conf.THRESH != 0.5:
+        string += f" thr: {conf.THRESH}"
+
+    if not train_dates:
+        labels = None
+    else:
+        labels = [
+            string.format(
+                *df.loc[df["training_date"] == d, display_keys].values[0]
+            )
+            for d in train_dates
+        ]
+
+    training_runs = []
+    for i, train in enumerate(train_dates):
+        training_runs += [Path(f"../trainings/{train}")]
+        for _ in range(
+            len(list(Path(f"../trainings/{train}").glob("unfreeze*")))
+        ):
+            labels += labels[i]
+    val_data = run_data_pipeline(
+        val_set, "val", return_spec=False, return_meta=True
+    )
+
+    model_name = [
+        df.loc[df["training_date"] == d, "Model"].values[0]
+        for d in train_dates
+    ]
+    keras_mod_name = [
+        df.loc[df["training_date"] == d, "keras_mod_name"].values[0]
+        for d in train_dates
+    ]
+
+    time_start = time.strftime("%Y%m%d_%H%M%S", time.gmtime())
+
+    if plot_metrics:
+        fig = plt.figure(constrained_layout=True, figsize=(6, 6))
+        subfigs = fig.subfigures(2, 1)  # , wspace=0.07, width_ratios=[1, 1])
+        plot_model_results(
+            train_dates, labels, fig=subfigs[0], legend=False
+        )  # , **info_dict)
+        eval_fig = subfigs[1]
+    else:
+        fig = plt.figure(constrained_layout=True, figsize=(5, 5))
+        eval_fig = fig
+        display_keys = ["keras_mod_name"]
+        table_df = df.loc[train_dates, display_keys]
+        if not len(table_df) == 0:
+            table_df.iloc[-1] = "GoogleModel"
+    plot_evaluation_metric(
+        model_name,
+        training_runs,
+        val_data,
+        plot_labels=labels,
+        fig=eval_fig,
+        plot_pr=True,
+        plot_cm=False,
+        titles=titles,
+        train_dates=train_dates,
+        label=None,
+        legend=False,
+        keras_mod_name=keras_mod_name,
+    )
+    fig.savefig(
+        f"../trainings/2022-11-30_01/{time_start}_results_combo.png", dpi=150
+    )
+
+
+def create_incorrect_prd_plot(
+    model_instance, train_date, val_data_path, **kwargs
+):
+    training_run = Path(f"../trainings/{train_date}").glob("unfreeze*")
+    val_data = run_data_pipeline(val_data_path, "val", return_spec=False)
+    labels, preds = get_labels_and_preds(
+        model_instance, training_run, val_data, **kwargs
+    )
+    preds = preds.reshape([len(preds)])
+    bin_preds = list(map(lambda x: 1 if x >= conf.THRESH else 0, preds))
+    false_pos, false_neg = [], []
+    for i in range(len(preds)):
+        if bin_preds[i] == 0 and labels[i] == 1:
+            false_neg.append(i)
+        if bin_preds[i] == 1 and labels[i] == 0:
+            false_pos.append(i)
+
+    offset = min([false_neg[0], false_pos[0]])
+    val_data = run_data_pipeline(
+        val_data_path, "val", return_spec=False, return_meta=True
+    )
+    val_data = val_data.batch(1)
+    val_data = val_data.map(lambda x, y, z, w: (spec()(x), y, z, w))
+    val_data = val_data.unbatch()
+    data = list(val_data.skip(offset))
+    fp = [data[i - offset] for i in false_pos]
+    fn = [data[i - offset] for i in false_neg]
+    plot_sample_spectrograms(
+        fn, dir=train_date, name=f"False_Negative", plot_meta=True, **kwargs
+    )
+    plot_sample_spectrograms(
+        fp, dir=train_date, name=f"False_Positive", plot_meta=True, **kwargs
+    )
+
+
+def create_table_plot():
+    time_start = time.strftime("%Y%m%d_%H%M%S", time.gmtime())
+    df = pd.read_csv("../trainings/20221124_meta_trainings.csv")
+    df.index = df["training_date"]
+    display_keys = ["keras_mod_name"]
+    col_labels = ["model name"]
+    table_df = df.loc[train_dates, display_keys]
+    table_df.iloc[-1] = "GoogleModel"
+    f, ax_tb = plt.subplots()
+    bbox = [0, 0, 1, 1]
+    ax_tb.axis("off")
+    font_size = 20
+    import seaborn as sns
+
+    color = list(sns.color_palette())
+    mpl_table = ax_tb.table(
+        cellText=table_df.values,
+        rowLabels=["     "] * len(table_df),
+        bbox=bbox,
+        colLabels=col_labels,
+        rowColours=color,
+    )
+    mpl_table.auto_set_font_size(False)
+    mpl_table.set_fontsize(font_size)
+    f.tight_layout()
+    f.savefig(
+        f"../trainings/{train_dates[-1]}/{time_start}_results_table.png",
+        dpi=150,
+    )
+
+
+if __name__ == "__main__":
+    tfrec_path = list(Path(conf.TFREC_DESTINATION).iterdir())
+    train_dates = ["2022-11-30_01"]
+
+    display_keys = ["Model", "keras_mod_name", "epochs", "init_lr", "final_lr"]
+
+    create_overview_plot(
+        train_dates,
+        tfrec_path,
+        display_keys,
+        plot_metrics=False,
+        titles=["all_data"],
+    )

acodet/front_end/help_strings.py

@@ -0,0 +1,65 @@
+RUN_OPTION = (
+    "Choose what option to run (currently only Inference is supported)."
+)
+SELECT_PRESET = "Based on the preset, different computations will run."
+SAMPLE_RATE = """
+If you need to change this, make sure that the sample rate 
+you set is below the sample rate of your audio files
+"""
+
+ENTER_PATH = """
+Enter the path to the directory containing the dataset(s) (so one above it). 
+The dropdown below will then allow you to choose the dataset(s) you would like to annotate on.
+"""
+
+CHOOSE_FOLDER = """
+Choose the folder containing the dataset(s) you would like to annotate on.
+"""
+THRESHOLD = "The threshold will be used to filter the predictions."
+ANNOTATIONS_DEFAULT_LOCATION = """
+The annotations are stored in this folder by default. If you want to specify another location, choose "No".
+"""
+ANNOTATIONS_TIMESTAMP_FOLDER = """
+Specify custom string to append to timestamp for foldername.\n
+Example: 2024-01-27_22-24-23___Custom-Dataset-1"""
+ANNOTATIONS_TIMESTAMP_RADIO = """
+By default the annotations folder is named according to the timestamp when it was created.
+By clicking Yes you can add a custom string to make it more specific.
+"""
+CHOOSE_TIMESTAMP_FOLDER = """
+These are the time stamps corresponding to computations that 
+have been performed on the machine previously. They all contain annotations files
+and can be used to filter the annotations with different thresholds or to generate
+hourly predictions.
+"""
+
+MULTI_DATA = """
+Are there multiple datasets located in the selected folder and would you
+like for all of them to be processed? If so select yes, if not, please only
+select the desired folder.
+"""
+
+SAVE_SELECTION_BTN = """
+By clicking, the selection tables of the chosen datasets will be 
+recomputed with the limit settings chosen above and saved in the same location
+with a name corresponding to the limit name and threshold."""
+
+LIMIT = """
+Choose between Simple and Sequence limit. Simple limit will only count the
+annotations that are above the threshold. Sequence limit will only count the
+annotations that are above the threshold and exceed the limit within 20
+consecutive sections.
+"""
+
+ANNOT_FILES_DROPDOWN = """
+Choose the annotation file you would like to inspect.
+"""
+
+SC_LIMIT = """
+The limit will be used to filter the predictions. The limit is the number of
+vocalizations within 20 consecutive sections that need to exceed the threshold.
+Higher limits mean less false positives but more false negatives. 
+Play around withit and see how the plot changes. 
+The idea behind this is to be able to tune the sensitivity of the model
+to the noise environment within the dataset.
+"""

acodet/front_end/st_annotate.py

@@ -0,0 +1,262 @@
+import streamlit as st
+from acodet.front_end import utils
+from pathlib import Path
+from acodet import create_session_file
+from acodet.front_end import help_strings
+
+
+session_config = create_session_file.read_session_file()
+
+
+def initial_dropdown(key):
+    """
+    Show dropdown menu to select what preset to run.
+
+    Parameters
+    ----------
+    key : string
+        unique identifier for streamlit objects
+
+    Returns
+    -------
+    int
+        preset number
+    """
+    return int(
+        st.selectbox(
+            "What predefined Settings would you like to run?",
+            (
+                "1 - generate new annotations",
+                "2 - filter existing annotations with different threshold",
+                "3 - generate hourly predictions",
+                "0 - all of the above",
+            ),
+            key=key,
+            help=help_strings.SELECT_PRESET,
+        )[0]
+    )
+
+
+class PresetInterfaceSettings:
+    def __init__(self, config, key) -> None:
+        """
+        Preset settings class. All methods are relevant for displaying
+        streamlit objects that will then be used to run the computations.
+
+        Parameters
+        ----------
+        config : dict
+            config dictionsary
+        key : string
+            unique identifier
+        """
+        self.config = config
+        self.key = key
+
+    def custom_timestamp_dialog(self):
+        """
+        Show radio buttons asking for custom folder names. If Yes is selected
+        allow user input that will be appended to timestamp for uses to give
+        custom names to annotation sessions.
+        """
+        timestamp_radio = st.radio(
+            f"Would you like to customize the annotaitons folder?",
+            ("No", "Yes"),
+            key="radio_" + self.key,
+            horizontal=True,
+            help=help_strings.ANNOTATIONS_TIMESTAMP_RADIO,
+        )
+        if timestamp_radio == "Yes":
+            self.config["annots_timestamp_folder"] = "___" + utils.user_input(
+                "Custom Folder string:",
+                "",
+                help=help_strings.ANNOTATIONS_TIMESTAMP_FOLDER,
+            )
+        elif timestamp_radio == "No":
+            self.config["annots_timestamp_folder"] = ""
+
+    def ask_to_continue_incomplete_inference(self):
+        continue_radio = st.radio(
+            f"Would you like to continue a cancelled session?",
+            ("No", "Yes"),
+            key="radio_continue_session_" + self.key,
+            horizontal=True,
+            help=help_strings.ANNOTATIONS_TIMESTAMP_RADIO,
+        )
+        if continue_radio == "Yes":
+            past_sessions = list(
+                Path(session_config["generated_annotations_folder"]).rglob(
+                    Path(self.config["sound_files_source"]).stem
+                )
+            )
+            if len(past_sessions) == 0:
+                st.text(
+                    f"""Sorry, but no annotations have been found in 
+                        `{session_config['generated_annotations_folder']}` on the currently
+                        selected dataset (`{self.config['sound_files_source']}`)."""
+                )
+            else:
+                previous_session = st.selectbox(
+                    "Which previous session would you like to continue?",
+                    past_sessions,
+                    key="continue_session_" + self.key,
+                    help=help_strings.SELECT_PRESET,
+                )
+                self.config["timestamp_folder"] = previous_session
+        else:
+            return True
+
+    def perform_inference(self):
+        """
+        Interface options when inference is selected, i.e. preset options 0 or 1.
+        """
+        path = st.text_input(
+            "Enter the path to your sound data:",
+            "tests/test_files",
+            help=help_strings.ENTER_PATH,
+        )
+        self.config["sound_files_source"] = utils.open_folder_dialogue(
+            path, key="folder_" + self.key, help=help_strings.CHOOSE_FOLDER
+        )
+        self.config["thresh"] = utils.validate_float(
+            utils.user_input(
+                "Model threshold:", "0.9", help=help_strings.THRESHOLD
+            )
+        )
+        self.advanced_settings()
+
+    def advanced_settings(self):
+        """
+        Expandable section showing advanced settings options.
+        """
+        with st.expander(r"**Advanced Settings**"):
+            continue_session = self.ask_to_continue_incomplete_inference()
+
+            if continue_session:
+                self.custom_timestamp_dialog()
+
+            self.ask_for_multiple_datasets()
+
+    def ask_for_multiple_datasets(self):
+        multiple_datasets = st.radio(
+            "Would you like to process multiple datasets in this session?",
+            ("No", "Yes"),
+            key=f"multi_datasets_{self.key}",
+            horizontal=True,
+            help=help_strings.MULTI_DATA,
+        )
+        if multiple_datasets == "Yes":
+            self.config["multi_datasets"] = True
+
+
+    def rerun_annotations(self):
+        """
+        Show options for rerunning annotations and saving the
+        selection tables with a different limit.
+        """
+        self.select_annotation_source_directory()
+        self.limit = st.radio(
+            "What limit would you like to set?",
+            ("Simple limit", "Sequence limit"),
+            key=f"limit_selec_{self.key}",
+            help=help_strings.LIMIT,
+        )
+
+        self.lim_obj = utils.Limits(self.limit, self.key)
+        self.lim_obj.create_limit_sliders()
+        self.lim_obj.save_selection_tables_with_limit_settings()
+
+    def select_annotation_source_directory(self):
+        """
+        Streamlit objects for preset options 2 and 3.
+        """
+        default_path = st.radio(
+            f"""The annotations I would like to filter are located in 
+            `{Path(session_config['generated_annotations_folder']).resolve()}`:""",
+            ("Yes", "No"),
+            key="radio_" + self.key,
+            horizontal=True,
+            help=help_strings.ANNOTATIONS_DEFAULT_LOCATION,
+        )
+        if default_path == "Yes":
+            self.config[
+                "generated_annotation_source"
+            ] = utils.open_folder_dialogue(
+                key="folder_default_" + self.key,
+                label="From the timestamps folders, choose the one you would like to work on.",
+                help=help_strings.CHOOSE_TIMESTAMP_FOLDER,
+                filter_existing_annotations=True,
+            )
+        elif default_path == "No":
+            path = st.text_input(
+                "Enter the path to your annotation data:",
+                "tests/test_files",
+                help=help_strings.ENTER_PATH,
+            )
+            self.config[
+                "generated_annotation_source"
+            ] = utils.open_folder_dialogue(
+                path,
+                key="folder_" + self.key,
+                label="From the timestamps folders, choose the one you would like to work on.",
+                help=help_strings.CHOOSE_TIMESTAMP_FOLDER,
+                filter_existing_annotations=True,
+            )
+            if (
+                Path(self.config["generated_annotation_source"]).stem
+                + Path(self.config["generated_annotation_source"]).suffix
+                == f"thresh_{session_config['default_threshold']}"
+            ):
+                st.write(
+                    """
+                        <p style="color:red; font-size:14px;">
+                        Please choose the top-level folder (usually a timestamp) instead.
+                        </p>""",
+                    unsafe_allow_html=True,
+                )
+
+
+def annotate_options(key="annot"):
+    """
+    Caller function for inference settings. Calls all necessary components
+    to show streamlit objects where users can choose what settings to run.
+
+    Parameters
+    ----------
+    key : str, optional
+        unique identifier for streamlit objects, by default "annot"
+
+    Returns
+    -------
+    boolean
+        True once all settings have been entered
+    """
+    preset_option = initial_dropdown(key)
+
+    st.session_state.preset_option = preset_option
+    utils.make_nested_btn_false_if_dropdown_changed(1, preset_option, 1)
+    utils.make_nested_btn_false_if_dropdown_changed(
+        run_id=1, preset_id=preset_option, btn_id=4
+    )
+    utils.next_button(id=1)
+
+    if not st.session_state.b1:
+        pass
+    else:
+        config = dict()
+        config["predefined_settings"] = preset_option
+        interface_settings = PresetInterfaceSettings(config, key)
+
+        if preset_option == 1 or preset_option == 0:
+            interface_settings.perform_inference()
+
+        elif preset_option == 2:
+            interface_settings.rerun_annotations()
+
+        elif preset_option == 3:
+            interface_settings.select_annotation_source_directory()
+            interface_settings.ask_for_multiple_datasets()
+
+        for k, v in interface_settings.config.items():
+            utils.write_to_session_file(k, v)
+        return True

acodet/front_end/st_generate_data.py

@@ -0,0 +1,115 @@
+import streamlit as st
+from acodet.front_end import utils
+
+
+def generate_data_options(key="gen_data"):
+    preset_option = int(
+        st.selectbox(
+            "How would you like run the program?",
+            (
+                "1 - generate new training data from reviewed annotations",
+                "2 - generate new training data from reviewed annotations "
+                "and fill space between annotations with noise annotations",
+            ),
+            key=key,
+        )[0]
+    )
+    st.session_state.preset_option = preset_option
+    utils.make_nested_btn_false_if_dropdown_changed(2, preset_option, 2)
+    utils.make_nested_btn_false_if_dropdown_changed(
+        run_id=1, preset_id=preset_option, btn_id=4
+    )
+    utils.next_button(id=2)
+    if not st.session_state.b2:
+        pass
+    else:
+        config = dict()
+        config["predefined_settings"] = preset_option
+
+        if preset_option == 1:
+            path1_sound = st.text_input(
+                "Enter the path to your sound data:", "."
+            )
+            config["sound_files_source"] = utils.open_folder_dialogue(
+                path1_sound, key="source_folder_" + key
+            )
+            path2_annots = st.text_input(
+                "Enter the path to your reviewed annotations:", "."
+            )
+            config["reviewed_annotation_source"] = utils.open_folder_dialogue(
+                path2_annots, key="reviewed_annotations_folder" + key
+            )
+
+            st.markdown("### Settings")
+            st.markdown("#### Audio preprocessing")
+            config["sample_rate"] = utils.validate_int(
+                utils.user_input("sample rate [Hz]", "2000")
+            )
+            config["context_window_in_seconds"] = utils.validate_float(
+                utils.user_input("context window length [s]", "3.9")
+            )
+            st.markdown("#### Spectrogram settings")
+            config["stft_frame_len"] = utils.validate_int(
+                utils.user_input("STFT frame length [samples]", "1024")
+            )
+            config["number_of_time_bins"] = utils.validate_int(
+                utils.user_input("number of time bins", "128")
+            )
+            st.markdown("#### TFRecord creationg settings")
+            config["tfrecs_limit_per_file"] = utils.validate_int(
+                utils.user_input(
+                    "limit of context windows per tfrecord file", "600"
+                )
+            )
+            config["train_ratio"] = utils.validate_float(
+                utils.user_input("trainratio", "0.7")
+            )
+            config["test_val_ratio"] = utils.validate_float(
+                utils.user_input("test validation ratio", "0.7")
+            )
+
+        elif preset_option == 2:
+            path1_sound = st.text_input(
+                "Enter the path to your sound data:", "."
+            )
+            config["sound_files_source"] = utils.open_folder_dialogue(
+                path1_sound, key="source_folder_" + key
+            )
+            path2_annots = st.text_input(
+                "Enter the path to your reviewed annotations:", "."
+            )
+            config["reviewed_annotation_source"] = utils.open_folder_dialogue(
+                path2_annots, key="reviewed_annotations_folder" + key
+            )
+
+            st.markdown("### Settings")
+            st.markdown("#### Audio preprocessing")
+            config["sample_rate"] = utils.validate_int(
+                utils.user_input("sample rate [Hz]", "2000")
+            )
+            config["context_window_in_seconds"] = utils.validate_float(
+                utils.user_input("context window length [s]", "3.9")
+            )
+            st.markdown("#### Spectrogram settings")
+            config["stft_frame_len"] = utils.validate_int(
+                utils.user_input("STFT frame length [samples]", "1024")
+            )
+            config["number_of_time_bins"] = utils.validate_int(
+                utils.user_input("number of time bins", "128")
+            )
+            st.markdown("#### TFRecord creationg settings")
+            config["tfrecs_limit_per_file"] = utils.validate_int(
+                utils.user_input(
+                    "limit of context windows per tfrecord file", "600"
+                )
+            )
+            config["train_ratio"] = utils.validate_float(
+                utils.user_input("trainratio", "0.7")
+            )
+            config["test_val_ratio"] = utils.validate_float(
+                utils.user_input("test validation ratio", "0.7")
+            )
+
+        for k, v in config.items():
+            utils.write_to_session_file(k, v)
+        return True

acodet/front_end/st_train.py

@@ -0,0 +1,92 @@
+import streamlit as st
+from acodet.front_end import utils
+
+
+def train_options(key="train"):
+    preset_option = int(
+        st.selectbox(
+            "How would you like run the program?",
+            (
+                "1 - train new model",
+                "2 - continue training on existing model and save model in the end",
+                "3 - evaluate saved model",
+                "4 - evaluate model checkpoint",
+                "5 - save model specified in advanced config",
+            ),
+            key=key,
+        )[0]
+    )
+    st.session_state.preset_option = preset_option
+    utils.make_nested_btn_false_if_dropdown_changed(3, preset_option, 3)
+    utils.make_nested_btn_false_if_dropdown_changed(
+        run_id=1, preset_id=preset_option, btn_id=4
+    )
+    utils.next_button(id=3)
+    if not st.session_state.b3:
+        pass
+    else:
+        config = dict()
+        config["predefined_settings"] = preset_option
+
+        if preset_option == 1:
+            st.markdown("### Model training settings")
+            st.markdown("#### Model architecture")
+            model_architecture = utils.user_dropdown(
+                "Which model architecture would you like to use?",
+                (
+                    "HumpBackNorthAtlantic",
+                    "ResNet50",
+                    "ResNet101",
+                    "ResNet152",
+                    "MobileNet",
+                    "DenseNet169",
+                    "DenseNet201",
+                    "EfficientNet0",
+                    "EfficientNet1",
+                    "EfficientNet2",
+                    "EfficientNet3",
+                    "EfficientNet4",
+                    "EfficientNet5",
+                    "EfficientNet6",
+                    "EfficientNet7",
+                ),
+            )
+            config["ModelClassName"] = model_architecture
+            if not model_architecture == "HumpBackNorthAtlantic":
+                config["keras_mod_name"] = True
+            st.markdown("#### Hyperparameters")
+            config["batch_size"] = utils.validate_int(
+                utils.user_input("batch size", "32")
+            )
+            config["epochs"] = utils.validate_int(
+                utils.user_input("epochs", "50")
+            )
+            config["steps_per_epoch"] = utils.validate_int(
+                utils.user_input("steps per epoch", "1000")
+            )
+            config["init_lr"] = utils.validate_float(
+                utils.user_input("initial learning rate", "0.0005")
+            )
+            config["final_lr"] = utils.validate_float(
+                utils.user_input("final learning rate", "0.000005")
+            )
+            st.markdown("#### Augmentations")
+            config["time_augs"] = utils.user_dropdown(
+                "Use time-shift augmentation", ("True", "False")
+            )
+            config["mixup_augs"] = utils.user_dropdown(
+                "Use mixup augmentation", ("True", "False")
+            )
+            config["spec_aug"] = utils.user_dropdown(
+                "Use specaugment augmentation", ("True", "False")
+            )
+
+            for key in ["time_augs", "mixup_augs", "spec_aug"]:
+                if config[key] == "False":
+                    config[key] = False
+                else:
+                    config[key] = True
+
+        for k, v in config.items():
+            utils.write_to_session_file(k, v)
+        return True

acodet/front_end/st_visualization.py

@@ -0,0 +1,286 @@
+import streamlit as st
+from pathlib import Path
+import pandas as pd
+import numpy as np
+from acodet.front_end import utils
+import plotly.express as px
+from acodet.create_session_file import read_session_file
+from acodet.front_end import help_strings
+
+conf = read_session_file()
+
+
+def output():
+    conf = read_session_file()
+    if st.session_state.run_option == 1:
+        if st.session_state.preset_option == 0:
+            disp = ShowAnnotationPredictions()
+            disp.show_annotation_predictions()
+            disp.create_tabs(
+                additional_headings=[
+                    "Filtered Files",
+                    "Annot. Plots",
+                    "Presence Plots",
+                ]
+            )
+            disp.show_stats()
+            disp.show_individual_files()
+            disp.show_individual_files(
+                tab_number=2, thresh_path=f"thresh_{conf['thresh']}"
+            )
+            plot_tabs = Results(disp)
+            plot_tabs.create_tabs()
+
+        elif st.session_state.preset_option == 1:
+            disp = ShowAnnotationPredictions()
+            disp.show_annotation_predictions()
+            disp.create_tabs()
+            disp.show_stats()
+            disp.show_individual_files()
+
+        elif st.session_state.preset_option == 2:
+            conf = read_session_file()
+            disp = ShowAnnotationPredictions()
+            disp.show_annotation_predictions()
+            disp.create_tabs()
+            disp.show_stats()
+            disp.show_individual_files(thresh_path=f"thresh_{conf['thresh']}")
+
+        elif st.session_state.preset_option == 3:
+            disp = ShowAnnotationPredictions()
+            disp.show_annotation_predictions()
+            disp.create_tabs(
+                additional_headings=[
+                    "Annot. Plots",
+                    "Presence Plots",
+                ]
+            )
+            disp.show_stats()
+            disp.show_individual_files()
+            plot_tabs = Results(disp, tab_number=2)
+            plot_tabs.create_tabs()
+
+
+class ShowAnnotationPredictions:
+    def show_annotation_predictions(self):
+        saved_annots_dir = Path(st.session_state.save_dir)
+        if len(list(saved_annots_dir.parents)) > 1:
+            self.annots_path = saved_annots_dir
+        else:
+            self.annots_path = Path(
+                conf["generated_annotations_folder"]
+            ).joinpath(saved_annots_dir)
+        st.markdown(
+            f"""Your annotations are saved in the folder: 
+            `{self.annots_path.resolve().as_posix()}`
+            """
+        )
+        utils.write_to_session_file(
+            "generated_annotation_source", str(self.annots_path)
+        )
+
+    def create_tabs(self, additional_headings=[]):
+        """
+        Create the tabs to display the respective results.
+
+        Parameters
+        ----------
+        additional_headings : list, optional
+            list of additional headings, by default []
+        """
+        tabs = st.tabs(
+            [
+                "Stats",
+                "Annot. Files",
+                *additional_headings,
+            ]
+        )
+        for i, tab in enumerate(tabs):
+            setattr(self, f"tab{i}", tab)
+
+    def show_stats(self):
+        """
+        Show stats file as pandas.DataFrame in a table.
+        """
+        with self.tab0:
+            try:
+                df = pd.read_csv(self.annots_path.joinpath("stats.csv"))
+                if "Unnamed: 0" in df.columns:
+                    df = df.drop(columns=["Unnamed: 0"])
+                st.dataframe(df, hide_index=True)
+            except Exception as e:
+                print(e)
+                st.write(
+                    """No stats.csv file found. Please run predefined settings 0, or 1 first
+                    to view this tab."""
+                )
+
+    def show_individual_files(
+        self, tab_number=1, thresh_path=conf["thresh_label"]
+    ):
+        with getattr(self, f"tab{tab_number}"):
+            path = self.annots_path.joinpath(thresh_path)
+            annot_files = [l for l in path.rglob("*.txt")]
+            display_annots = [
+                f.relative_to(path).as_posix() for f in annot_files
+            ]
+            chosen_file = st.selectbox(
+                label=f"""Choose a generated annotations file from 
+                `{path.resolve()}`""",
+                options=display_annots,
+                key=f"file_selec_{tab_number}",
+                help=help_strings.ANNOT_FILES_DROPDOWN,
+            )
+            st.write("All of these files can be imported into Raven directly.")
+            df = pd.read_csv(path.joinpath(chosen_file), sep="\t")
+            st.dataframe(df, hide_index=True)
+
+
+class Results(utils.Limits):
+    def __init__(self, disp_obj, tab_number=3) -> None:
+        """
+        Results class containing all of the data processings to prepare
+        data for plots and tables.
+
+        Parameters
+        ----------
+        disp_obj : object
+            ShowAnnotationPredictions object to link processing to
+            the respective streamlit widget
+        tab_number : int, optional
+            number of tab to show results in, by default 3
+        """
+        self.plots_paths = [
+            [d for d in p.iterdir() if d.is_dir()]
+            for p in disp_obj.annots_path.rglob("*analysis*")
+        ][0]
+        if not self.plots_paths:
+            st.write(
+                "No analysis files found for this dataset. "
+                "Please run predefined settings 0, or 1 first."
+            )
+            st.stop()
+        self.disp_obj = disp_obj
+        self.tab_number = tab_number
+
+    def create_tabs(self):
+        """
+        Create tabs for plots.
+        """
+        self.tabs = {
+            "binary": getattr(self.disp_obj, f"tab{self.tab_number}"),
+            "presence": getattr(self.disp_obj, f"tab{self.tab_number+1}"),
+        }
+        for key, tab in self.tabs.items():
+            self.init_tab(tab=tab, key=key)
+
+    def init_tab(self, tab, key):
+        with tab:
+            datasets = [l.stem for l in self.plots_paths]
+
+            chosen_dataset = st.selectbox(
+                label=f"""Choose a dataset:""",
+                options=datasets,
+                key=f"dataset_selec_{key}",
+            )
+            self.chosen_dataset = (
+                self.disp_obj.annots_path.joinpath(conf["thresh_label"])
+                .joinpath("analysis")
+                .joinpath(chosen_dataset)
+            )
+
+            limit = st.radio(
+                "What limit would you like to set?",
+                ("Simple limit", "Sequence limit"),
+                key=f"limit_selec_{key}",
+                help=help_strings.LIMIT,
+            )
+
+            super(Results, self).__init__(limit, key)
+
+            results = PlotDisplay(self.chosen_dataset, tab, key)
+            results.plot_df(self.limit_label)
+
+            self.create_limit_sliders()
+            self.rerun_computation_btn()
+
+            self.save_selection_tables_with_limit_settings()
+
+    def rerun_computation_btn(self):
+        """
+        Show rerun computation button after limits have been set and
+        execute run.main.
+        """
+        rerun = st.button("Rerun computation", key=f"update_plot_{self.key}")
+        st.session_state.progbar_update = st.progress(0, text="Updating plot")
+        if rerun:
+            utils.write_to_session_file(self.thresh_label, self.thresh)
+            if self.sc:
+                utils.write_to_session_file(self.limit_label, self.limit)
+
+            import run
+
+            run.main(
+                dont_save_plot=True,
+                sc=self.sc,
+                fetch_config_again=True,
+                preset=3,
+                update_plot=True,
+            )
+
+
+class PlotDisplay:
+    def __init__(self, chosen_dataset, tab, key) -> None:
+        self.chosen_dataset = chosen_dataset
+        self.tab = tab
+        self.key = key
+
+        if key == "binary":
+            self.path_prefix = "hourly_annotation"
+            self.cbar_label = "Number of annotations"
+            self.c_range = [0, conf["max_annots_per_hour"]]
+        elif key == "presence":
+            self.path_prefix = "hourly_presence"
+            self.cbar_label = "Presence"
+            self.c_range = [0, 1]
+
+    def plot_df(self, limit_label):
+        """
+
+        Plot dataframe showing either hourly presence or annotation count
+        in an interactive plotly visualization.
+
+        TODO onclick display of scrollable spectrogram would be really sick.
+
+        Parameters
+        ----------
+        limit_label : string
+            key of config dict to acces simple or sequence limit
+        """
+        df = pd.read_csv(
+            self.chosen_dataset.joinpath(
+                f"{self.path_prefix}_{limit_label}.csv"
+            )
+        )
+        df.index = pd.DatetimeIndex(df.Date)
+        df = df.reindex(
+            pd.date_range(df.index[0], df.index[-1]), fill_value=np.nan
+        )
+
+        h_of_day_str = ["%.2i:00" % i for i in range(24)]
+        h_pres = df.loc[:, h_of_day_str]
+
+        fig = px.imshow(
+            h_pres.T,
+            labels=dict(x="Date", y="Time of Day", color=self.cbar_label),
+            x=df.index,
+            y=h_of_day_str,
+            range_color=self.c_range,
+            color_continuous_scale="blugrn",
+        )
+
+        fig.update_xaxes(showgrid=False)
+        fig.update_yaxes(showgrid=False)
+        fig.update_layout(hovermode="x")
+
+        st.plotly_chart(fig)

acodet/front_end/utils.py

@@ -0,0 +1,238 @@
+import streamlit as st
+from pathlib import Path
+from acodet import create_session_file
+from . import help_strings
+
+conf = create_session_file.read_session_file()
+import json
+import keras
+
+
+def open_folder_dialogue(
+    path=conf["generated_annotations_folder"],
+    key="folder",
+    label="Choose a folder",
+    filter_existing_annotations=False,
+    **kwargs,
+):
+    try:
+        if not filter_existing_annotations:
+            foldernames_list = [
+                f"{x.stem}{x.suffix}"
+                for x in Path(path).iterdir()
+                if x.is_dir()
+            ]
+            if f"thresh_{conf['default_threshold']}" in foldernames_list:
+                foldernames_list = [f"thresh_{conf['default_threshold']}"]
+        else:
+            foldernames_list = [
+                f"{x.stem}{x.suffix}"
+                for x in Path(path).iterdir()
+                if x.is_dir()
+            ]
+            foldernames_list.sort()
+            foldernames_list.reverse()
+        # create selectbox with the foldernames
+        chosen_folder = st.selectbox(
+            label=label, options=foldernames_list, key=key, **kwargs
+        )
+
+        # set the full path to be able to refer to it
+        directory_path = Path(path).joinpath(chosen_folder)
+        return str(directory_path)
+
+    except FileNotFoundError:
+        st.write("Folder does not exist, retrying.")
+
+
+def next_button(id, text="Next", **kwargs):
+    if f"b{id}" not in st.session_state:
+        setattr(st.session_state, f"b{id}", False)
+    val = st.button(text, key=f"button_{id}", **kwargs)
+    if val:
+        setattr(st.session_state, f"b{id}", True)
+        make_nested_btns_false_on_click(id)
+        if text in ["Run computations", "Next"]:
+            st.session_state.run_finished = False
+
+
+def user_input(label, val, **input_params):
+    c1, c2 = st.columns(2)
+    c1.markdown("##")
+    input_params.setdefault("key", label)
+    c1.markdown(label)
+    return c2.text_input(" ", val, **input_params)
+
+
+def user_dropdown(label, vals, **input_params):
+    c1, c2 = st.columns(2)
+    c1.markdown("##")
+    input_params.setdefault("key", label)
+    c1.markdown(label)
+    return c2.selectbox(" ", vals, **input_params)
+
+
+def write_to_session_file(key, value):
+    if "session_started" in st.session_state:
+        setattr(st.session_state, key, value)
+    else:
+        with open("acodet/src/tmp_session.json", "r") as f:
+            session = json.load(f)
+        session[key] = value
+        with open("acodet/src/tmp_session.json", "w") as f:
+            json.dump(session, f)
+
+
+def validate_float(input):
+    try:
+        return float(input)
+    except ValueError:
+        st.write(
+            '<p style="color:red; font-size:10px;">The value you entered is not a number.</p>',
+            unsafe_allow_html=True,
+        )
+
+
+def validate_int(input):
+    try:
+        return int(input)
+    except ValueError:
+        st.write(
+            '<p style="color:red; font-size:12px;">The value you entered is not a number.</p>',
+            unsafe_allow_html=True,
+        )
+
+
+def make_nested_btn_false_if_dropdown_changed(run_id, preset_id, btn_id):
+    if "session_started" in st.session_state:
+        session = {**st.session_state}
+    else:
+        with open("acodet/src/tmp_session.json", "r") as f:
+            session = json.load(f)
+    if not (
+        session["run_config"] == run_id
+        and session["predefined_settings"] == preset_id
+    ):
+        setattr(st.session_state, f"b{btn_id}", False)
+
+
+def make_nested_btns_false_on_click(btn_id):
+    btns = [i for i in range(btn_id + 1, 6)]
+    for btn in btns:
+        setattr(st.session_state, f"b{btn}", False)
+
+
+def prepare_run():
+    if st.session_state.run_option == 1:
+        st.markdown("""---""")
+        st.markdown("## Computation started, please wait.")
+        if st.session_state.preset_option in [0, 1]:
+            kwargs = {
+                "callbacks": TFPredictProgressBar,
+                "progbar1": st.progress(0, text="Current file"),
+                "progbar2": st.progress(0, text="Overall progress"),
+            }
+        else:
+            kwargs = {"progbar1": st.progress(0, text="Progress")}
+    return kwargs
+
+
+class Limits:
+    def __init__(self, limit, key):
+        """
+        A simple class to contain all methods revolving around the limit sliders
+        for simple and sequence limit.
+
+        Parameters
+        ----------
+        limit : string
+            either simple or sequence limit, from radio btn
+        key : string
+            unique identifier for streamlit options
+        """
+        self.key = "limit_" + key
+        self.save_btn = False
+        if limit == "Simple limit":
+            self.limit_label = "simple_limit"
+            self.thresh_label = "thresh"
+            self.sc = False
+            self.limit_max = 50
+        elif limit == "Sequence limit":
+            self.limit_label = "sequence_limit"
+            self.thresh_label = "sequence_thresh"
+            self.sc = True
+            self.limit_max = 20
+
+    def create_limit_sliders(self):
+        """
+        Show sliders for simple and sequence limit, depending on the selection
+        of the radio btn.
+        """
+        self.thresh = st.slider(
+            "Threshold",
+            0.35,
+            0.99,
+            conf[self.thresh_label],
+            0.01,
+            key=f"thresh_slider_{self.key}",
+            help=help_strings.THRESHOLD,
+        )
+
+        if self.sc:
+            self.limit = st.slider(
+                "Limit",
+                1,
+                self.limit_max,
+                conf[self.limit_label],
+                1,
+                key=f"limit_slider_{self.key}",
+                help=help_strings.SC_LIMIT,
+            )
+
+    def show_save_selection_tables_btn(self):
+        """Show save selection tables btn."""
+        self.save_btn = st.button(
+            "Save tables", self.key, help=help_strings.SAVE_SELECTION_BTN
+        )
+
+    def save_selection_tables_with_limit_settings(self):
+        """
+        Save the selection tables of the chosen dataset again with
+        the selected settings of the respective limit.
+        """
+        self.show_save_selection_tables_btn()
+        if self.save_btn:
+            st.session_state.progbar_update = st.progress(0, text="Progress")
+            write_to_session_file(self.thresh_label, self.thresh)
+            if self.sc:
+                write_to_session_file(self.limit_label, self.limit)
+
+            import run
+
+            run.main(
+                dont_save_plot=True,
+                sc=self.sc,
+                fetch_config_again=True,
+                preset=3,
+                save_filtered_selection_tables=True,
+            )
+
+
+class TFPredictProgressBar(keras.callbacks.Callback):
+    def __init__(self, num_of_files, progbar1, progbar2, **kwargs):
+        self.num_of_files = num_of_files
+        self.pr_bar1 = progbar1
+        self.pr_bar2 = progbar2
+
+    def on_predict_end(self, logs=None):
+        self.pr_bar2.progress(
+            st.session_state.progbar1 / self.num_of_files,
+            text="Overall progress",
+        )
+
+    def on_predict_batch_begin(self, batch, logs=None):
+        if self.params["steps"] == 1:
+            denominator = 1
+        else:
+            denominator = self.params["steps"] - 1
+        self.pr_bar1.progress(batch / denominator, text="Current file")

acodet/funcs.py

@@ -0,0 +1,747 @@
+from email import generator
+import re
+import zipfile
+import datetime as dt
+import json
+import tensorflow as tf
+import numpy as np
+import librosa as lb
+from pathlib import Path
+import pandas as pd
+from . import global_config as conf
+
+############# ANNOTATION helpers ############################################
+
+
+def remove_str_flags_from_predictions(df):
+    # TODO wenn annotation_column nicht in columns ist fehler raisen
+    n = df.loc[df[conf.ANNOTATION_COLUMN] == "n"].index
+    n_ = df.loc[df[conf.ANNOTATION_COLUMN] == "n "].index
+    u = df.loc[df[conf.ANNOTATION_COLUMN] == "u"].index
+    u_ = df.loc[df[conf.ANNOTATION_COLUMN] == "u "].index
+    c = df.loc[df[conf.ANNOTATION_COLUMN] == "c"].index
+    c_ = df.loc[df[conf.ANNOTATION_COLUMN] == "c "].index
+
+    clean = df.drop([*n, *u, *c, *n_, *u_, *c_])
+    clean.loc[:, conf.ANNOTATION_COLUMN] = clean[
+        conf.ANNOTATION_COLUMN
+    ].astype(float)
+    return clean
+
+
+############# TFRECORDS helpers #############################################
+def get_annots_for_file(annots: pd.DataFrame, file: str) -> pd.DataFrame:
+    """
+    Get annotations for a file and sort by the start time of the annotated
+    call.
+
+    Parameters
+    ----------
+    annots : pd.DataFrame
+        global annotations dataframe
+    file : str
+        file path
+
+    Returns
+    -------
+    pd.DataFrame
+        filtered annotations dataframe
+    """
+    return annots[annots.filename == file].sort_values("start")
+
+
+def get_dt_filename(file):
+    if isinstance(file, Path):
+        stem = file.stem
+    else:
+        stem = file
+
+    if "_annot_" in stem:
+        stem = stem.split("_annot_")[0]
+
+    numbs = re.findall("[0-9]+", stem)
+    numbs = [n for n in numbs if len(n) % 2 == 0]
+
+    i, datetime = 1, ""
+    while len(datetime) < 12:
+        if i > 1000:
+            raise NameError(
+                """Time stamp Error: time stamp in filename 
+                doesn't fit any known pattern. If you would
+                like to predict this file anyway, please
+                choose predefined settings option 1.
+                """
+            )
+        datetime = "".join(numbs[-i:])
+        i += 1
+
+    i = 1
+    while 12 <= len(datetime) > 14:
+        datetime = datetime[:-i]
+
+    for _ in range(2):
+        try:
+            if len(datetime) == 12:
+                file_date = dt.datetime.strptime(datetime, "%y%m%d%H%M%S")
+            elif len(datetime) == 14:
+                file_date = dt.datetime.strptime(datetime, "%Y%m%d%H%M%S")
+        except:
+            i = 1
+            while len(datetime) > 12:
+                datetime = datetime[:-i]
+
+    try:
+        # print(file_date)
+        return file_date
+    except Exception as e:
+        print(
+            "File date naming not understood.\n",
+            "This will be prevent hourly prediction computation.\n",
+            e,
+        )
+        return "ERROR"
+
+
+def get_channel(dir):
+    if "_CH" in str(dir)[-5:]:
+        channel = int(re.findall("[0-9]+", str(dir)[-5:])[0]) - 1
+    else:
+        channel = 0
+    return channel
+
+
+def load_audio(file, channel=0, **kwargs) -> np.ndarray:
+    """
+    Load audio file, print error if file is corrupted. If the sample rate
+    specified in the config file is not the same as the downsample sample
+    rate, resample the audio file accordingly.
+
+    Parameters
+    ----------
+    file : str or pathlib.Path
+        file path
+
+    Returns
+    -------
+    audio_flat: np.ndarray
+        audio array
+    """
+    try:
+        if conf.DOWNSAMPLE_SR and conf.SR != conf.DOWNSAMPLE_SR:
+            with open(file, "rb") as f:
+                audio_flat, _ = lb.load(
+                    f, sr=conf.DOWNSAMPLE_SR, mono=False, **kwargs
+                )[channel]
+            if len(audio_flat.shape) > 1:
+                audio_flat = audio_flat[channel]
+
+            audio_flat = lb.resample(
+                audio_flat, orig_sr=conf.DOWNSAMPLE_SR, target_sr=conf.SR
+            )
+        else:
+            with open(file, "rb") as f:
+                audio_flat, _ = lb.load(f, sr=conf.SR, mono=False, **kwargs)
+            if len(audio_flat.shape) > 1:
+                audio_flat = audio_flat[channel]
+
+        if len(audio_flat) == 0:
+            return
+        return audio_flat
+    except:
+        print("File is corrputed and can't be loaded.")
+        return
+
+
+def return_windowed_file(file) -> tuple([np.ndarray, np.ndarray]):
+    """
+    Load audio file and turn the 1D array into a 2D array. The rows length
+    corresponds to the window length specified by the config file. The
+    number of columns results from the division of the 1D array length
+    by the context window length. The incomplete remainder is discarded.
+    Along with the window, a 1D array of times is returned, corresponding
+    to the beginning in seconds of each context window within the original
+    file.
+
+    Parameters
+    ----------
+    file : str or pathlib.Path
+        file path
+
+    Returns
+    -------
+    audio_arr: np.ndarray
+        2D audio array
+    times: np.ndarray
+        start times of the context windows
+    """
+    audio = load_audio(file)
+    audio = audio[: len(audio) // conf.CONTEXT_WIN * conf.CONTEXT_WIN]
+    audio_arr = audio.reshape(
+        [len(audio) // conf.CONTEXT_WIN, conf.CONTEXT_WIN]
+    )
+
+    times = np.arange(
+        0,
+        audio_arr.shape[0] * conf.CONTEXT_WIN / conf.SR,
+        conf.CONTEXT_WIN / conf.SR,
+    )
+    return audio_arr, times
+
+
+def cntxt_wndw_arr(
+    annotations: pd.DataFrame, file, inbetween_noise: bool = True, **kwargs
+) -> tuple:
+    """
+    Load an audio file, with the duration given by the time difference
+    between the first and last annotation. Iterate through the annotations
+    and extract 1D segments from the audio array based on the annotations,
+    the sample rate and the context window length, all specified in the config
+    file. The resulting context window is appended to a list, yielding a 2D
+    array with context windows corresponding to the annotated sections.
+
+    To be able to find the section in the original audio file, another list
+    is filled with all the start times noted in the annotations multiplied
+    by the sample rate. The times list contains the beginning time of each
+    context window in samples.
+
+    Finally if the argument 'inbetween_noise' is False, the nosie arrays in between
+    the calls are collected and all 4 arrays are returned.
+
+    Parameters
+    ----------
+    annotations : pd.DataFrame
+        annotations of vocalizations in file
+    file : str or pathlib.Path
+        file path
+    inbetween_noise : bool, defaults to True
+        decide if in between noise should be collected as well or, in case
+        the argument is True, all the annotations are already all noise, in
+        which case no in between noise is returned
+
+    Returns
+    -------
+    seg_ar: np.ndarray
+        segment array containing the 2D audio array
+    noise_ar: np.ndarray
+        2D audio array of noise
+    times_c
+        time list for calls
+    times_n
+        time list for noise
+    """
+    duration = annotations["end"].iloc[-1] + conf.CONTEXT_WIN / conf.SR
+    audio = load_audio(file, duration=duration)
+
+    segs, times = [], []
+    for _, row in annotations.iterrows():
+        num_windows = round(
+            (row.end - row.start) / (conf.CONTEXT_WIN / conf.SR) - 1
+        )
+        num_windows = num_windows or 1
+        for i in range(
+            num_windows
+        ):  # TODO fuer grosse annotationen mehrere fenster erzeugen
+            start = row.start + i * (conf.CONTEXT_WIN / conf.SR)
+            beg = int(start * conf.SR)
+            end = int(start * conf.SR + conf.CONTEXT_WIN)
+
+            if len(audio[beg:end]) == conf.CONTEXT_WIN:
+                segs.append(audio[beg:end])
+                times.append(beg)
+            else:
+                end = len(audio)
+                beg = end - conf.CONTEXT_WIN
+                segs.append(audio[beg:end])
+                times.append(beg)
+                break
+
+    segs = np.array(segs, dtype="float32")
+    times = np.array(times, dtype="float32")
+
+    # TODO docstrings aufraumen
+    if len(segs) - len(annotations) < 0:
+        annotations = annotations.drop(
+            annotations.index[len(segs) - len(annotations) :]
+        )
+    if (
+        not inbetween_noise
+    ):  # TODO mismatch in lengths, allow longer active learning annots
+        seg_ar = np.array(segs[annotations["label"] == 1], dtype="float32")
+        times_c = np.array(times[annotations["label"] == 1], dtype="float32")
+    else:
+        seg_ar = segs
+        times_c = times
+    if inbetween_noise:
+        noise_ar, times_n = return_inbetween_noise_arrays(audio, annotations)
+    elif len(annotations.loc[annotations["label"] == 0]) > 0:
+        noise_ar = np.array(segs[annotations["label"] == 0], dtype="float32")
+        times_n = np.array(times[annotations["label"] == 0], dtype="float32")
+    else:
+        noise_ar, times_n = np.array([]), np.array([])
+
+    return seg_ar, noise_ar, times_c, times_n
+
+
+def wins_bet_calls(annotations: pd.DataFrame) -> list:
+    """
+    Returns a list of ints, corresponding to the number of context windows
+    that fit between calls. The indexing is crucial, so that each start time
+    is subtracted from the previous end time, thereby yielding the gap length.
+
+    Parameters
+    ----------
+    annotations : pd.DataFrame
+        annotations
+
+    Returns
+    -------
+    list
+        number of context windows that fit between the start of one and the end of
+        the previous annotation
+    """
+    beg_min_start = annotations.start[1:].values - annotations.end[:-1].values
+    return (beg_min_start // (conf.CONTEXT_WIN / conf.SR)).astype(int)
+
+
+def return_inbetween_noise_arrays(
+    audio: np.ndarray, annotations: pd.DataFrame
+) -> tuple:
+    """
+    Collect audio arrays based on the gaps between vocalizations.
+    Based on the number of context windows that fit inbetween two
+    subsequent annotations, the resulting amount of segments are
+    extracted from the audio file.
+    Again, a list containing the start time of each array is also
+    retrieved.
+    If no entire context window fits between two annotations, no
+    noise sample is generated.
+    The resulting 2D noise array is returned along with the times.
+
+    Parameters
+    ----------
+    audio : np.ndarray
+        flat 1D audio array
+    annotations : pd.DataFrame
+        annotations of vocalizations
+
+    Returns
+    -------
+    np.ndarray
+        2D audio array of noise
+    times: list
+        start times of each context window
+    """
+    noise_ar, times = list(), list()
+    for ind, num_wndws in enumerate(wins_bet_calls(annotations)):
+        if num_wndws < 1:
+            continue
+
+        for window_ind in range(num_wndws):
+            beg = (
+                int(annotations.end.iloc[ind] * conf.SR)
+                + conf.CONTEXT_WIN * window_ind
+            )
+            end = beg + conf.CONTEXT_WIN
+            noise_ar.append(audio[beg:end])
+            times.append(beg)
+
+    return np.array(noise_ar, dtype="float32"), times
+
+
+def get_train_set_size(tfrec_path):
+    if not isinstance(tfrec_path, list):
+        tfrec_path = [tfrec_path]
+    train_set_size, noise_set_size = 0, 0
+    for dataset_dir in tfrec_path:
+        try:
+            for dic in Path(dataset_dir).glob("**/*dataset*.json"):
+                with open(dic, "r") as f:
+                    data_dict = json.load(f)
+                    if "noise" in str(dic):
+                        noise_set_size += data_dict["dataset"]["size"]["train"]
+                    elif "train" in data_dict["dataset"]["size"]:
+                        train_set_size += data_dict["dataset"]["size"]["train"]
+        except:
+            print(
+                "No dataset dictionary found, estimating dataset size."
+                "WARNING: This might lead to incorrect learning rates!"
+            )
+            train_set_size += 5000
+            noise_set_size += 100
+    return train_set_size, noise_set_size
+
+
+################ Plotting helpers ###########################################
+
+
+def get_time(time: float) -> str:
+    """
+    Return time in readable string format m:s.ms.
+
+    Parameters
+    ----------
+    time : float
+        time in seconds
+
+    Returns
+    -------
+    str
+        time in minutes:seconds.miliseconds
+    """
+    return f"{int(time/60)}:{np.mod(time, 60):.1f}s"
+
+
+################ Model Training helpers #####################################
+
+
+def save_model_results(ckpt_dir: str, result: dict):
+    """
+    Format the results dict so that no error occurrs when saving the json.
+
+    Parameters
+    ----------
+    ckpt_dir : str
+        checkpoint path
+    result : dict
+        training results
+    """
+    result["fbeta"] = [float(n) for n in result["fbeta"]]
+    result["val_fbeta"] = [float(n) for n in result["val_fbeta"]]
+    result["fbeta1"] = [float(n) for n in result["fbeta1"]]
+    result["val_fbeta1"] = [float(n) for n in result["val_fbeta1"]]
+    with open(f"{ckpt_dir}/results.json", "w") as f:
+        json.dump(result, f)
+
+
+def get_val_labels(
+    val_data: tf.data.Dataset, num_of_samples: int
+) -> np.ndarray:
+    """
+    Return all validation set labels. The dataset is batched with the dataset
+    size, thus creating one batch from the entire dataset. This batched
+    dataset is then converted to a list and its numpy attribute is returned.
+
+    Parameters
+    ----------
+    val_data : tf.data.Dataset
+        validation set
+    num_of_samples : int
+        length of dataset
+
+    Returns
+    -------
+    np.ndarray
+        array of all validation set labels
+    """
+    return list(val_data.batch(num_of_samples))[0][1].numpy()
+
+
+############### Model Evaluation helpers ####################################
+
+
+def print_evaluation(
+    val_data: tf.data.Dataset, model: tf.keras.Sequential, batch_size: int
+):
+    """
+    Print evaluation results.
+
+    Parameters
+    ----------
+    val_data : tf.data.Dataset
+        validation data set
+    model : tf.keras.Sequential
+        keras model
+    batch_size : int
+        batch size
+    """
+    model.evaluate(val_data, batch_size=batch_size, verbose=2)
+
+
+def get_pr_arrays(
+    labels: np.ndarray, preds: np.ndarray, metric: str, **kwargs
+) -> np.ndarray:
+    """
+    Compute Precision or Recall on given set of labels and predictions.
+    Threshold values are created with 0.01 increments.
+
+    Parameters
+    ----------
+    labels : np.ndarray
+        labels
+    preds : np.ndarray
+        predictions
+    metric : str
+        Metric to calculate i.e. Recall or Precision
+
+    Returns
+    -------
+    np.ndarray
+        resulting values
+    """
+    r = getattr(tf.keras.metrics, metric)(**kwargs)
+    r.update_state(labels, preds.reshape(len(preds)))
+    return r.result().numpy()
+
+
+############## Generate Model Annotations helpers ############################
+
+
+def get_files(
+    *, location: str = f"{conf.GEN_ANNOTS_DIR}", search_str: str = "*.wav"
+) -> list:
+    """
+    Find all files corresponding to given search string within a specified
+    location.
+
+    Parameters
+    ----------
+    location : str, optional
+        root directory of files, by default 'generated_annotations/src'
+    search_str : str, optional
+        search string containing search pattern, for example '*.wav',
+        by default '*.wav'
+
+    Returns
+    -------
+    generator
+        list containing pathlib.Path objects of all files fitting
+        the pattern
+    """
+    folder = Path(location)
+    return list(folder.glob(search_str))
+
+
+def window_data_for_prediction(audio: np.ndarray) -> tf.Tensor:
+    """
+    Compute predictions based on spectrograms. First the number of context
+    windows that fit into the audio array are calculated. The result is an
+    integer unless the last section is reached, in that case the audio is
+    zero padded to fit the length of a multiple of the context window length.
+    The array is then zero padded to fit a integer multiple of the context
+    window.
+
+    Parameters
+    ----------
+    audio : np.ndarray
+        1D audio array
+
+    Returns
+    -------
+    tf.Tensor
+        2D audio tensor with shape [context window length, number of windows]
+    """
+    num = np.ceil(len(audio) / conf.CONTEXT_WIN)
+    # zero pad in case the end is reached
+    audio = [*audio, *np.zeros([int(num * conf.CONTEXT_WIN - len(audio))])]
+    wins = np.array(audio).reshape([int(num), conf.CONTEXT_WIN])
+
+    return tf.convert_to_tensor(wins)
+
+
+def create_Raven_annotation_df(preds: np.ndarray, ind: int) -> pd.DataFrame:
+    """
+    Create a DataFrame with column names according to the Raven annotation
+    format. The DataFrame is then filled with the corresponding values.
+    Beginning and end times for each context window, high and low frequency
+    (from config), and the prediction values. Based on the predicted values,
+    the sections with predicted labels of less than the threshold are
+    discarded.
+
+    Parameters
+    ----------
+    preds : np.ndarray
+        predictions
+    ind : int
+        batch of current predictions (in case predictions are more than
+        the specified limitation for predictions)
+
+    Returns
+    -------
+    pd.DataFrame
+        annotation dataframe for current batch, filtered by threshold
+    """
+    df = pd.DataFrame(
+        columns=[
+            "Begin Time (s)",
+            "End Time (s)",
+            "High Freq (Hz)",
+            "Low Freq (Hz)",
+        ]
+    )
+
+    df["Begin Time (s)"] = (
+        np.arange(0, len(preds)) * conf.CONTEXT_WIN
+    ) / conf.SR
+    df["End Time (s)"] = df["Begin Time (s)"] + conf.CONTEXT_WIN / conf.SR
+
+    df["Begin Time (s)"] += (ind * conf.PRED_BATCH_SIZE) / conf.SR
+    df["End Time (s)"] += (ind * conf.PRED_BATCH_SIZE) / conf.SR
+
+    df["High Freq (Hz)"] = conf.ANNOTATION_DF_FMAX
+    df["Low Freq (Hz)"] = conf.ANNOTATION_DF_FMIN
+    df[conf.ANNOTATION_COLUMN] = preds
+
+    return df.iloc[preds.reshape([len(preds)]) > conf.DEFAULT_THRESH]
+
+
+def create_annotation_df(
+    audio_batches: np.ndarray,
+    model: tf.keras.Sequential,
+    callbacks: None = None,
+    **kwargs,
+) -> pd.DataFrame:
+    """
+    Create a annotation dataframe containing all necessary information to
+    be imported into a annotation program. The loaded audio batches are
+    iterated over and used to predict labels. All information is then used
+    to fill a DataFrame. After having gone through all batches, the index
+    column is set to a increasing integers named 'Selection' (convention).
+
+    Parameters
+    ----------
+    audio_batches : np.ndarray
+        audio batches
+    model : tf.keras.Sequential
+        model instance to predict values
+
+    Returns
+    -------
+    pd.DataFrame
+        annotation dataframe
+    """
+    annots = pd.DataFrame()
+    for ind, audio in enumerate(audio_batches):
+        if callbacks is not None and ind == 0:
+            callbacks = callbacks(**kwargs)
+        preds = model.predict(
+            window_data_for_prediction(audio), callbacks=callbacks
+        )
+        df = create_Raven_annotation_df(preds, ind)
+        annots = pd.concat([annots, df], ignore_index=True)
+
+    annots.index = np.arange(1, len(annots) + 1)
+    annots.index.name = "Selection"
+    return annots
+
+
+def batch_audio(audio_flat: np.ndarray) -> np.ndarray:
+    """
+    Divide 1D audio array into batches depending on the config parameter
+    pred_batch_size (predictions batch size) i.e. the number of windows
+    that are being simultaneously predicted.
+
+    Parameters
+    ----------
+    audio_flat : np.ndarray
+        1D audio array
+
+    Returns
+    -------
+    np.ndarray
+        batched audio array
+    """
+    if len(audio_flat) < conf.PRED_BATCH_SIZE:
+        audio_batches = [audio_flat]
+    else:
+        n = conf.PRED_BATCH_SIZE
+        audio_batches = [
+            audio_flat[i : i + n]
+            for i in range(0, len(audio_flat), conf.PRED_BATCH_SIZE)
+        ]
+    return audio_batches
+
+
+def get_directory_structure_relative_to_config_path(file):
+    return file.relative_to(conf.SOUND_FILES_SOURCE).parent
+
+
+def get_top_dir_name_if_only_one_parent_dir(file, parent_dirs):
+    if str(parent_dirs) == ".":
+        parent_dirs = file.parent.stem
+    return parent_dirs
+
+
+def check_top_dir_crit(parent_dirs):
+    return Path(parent_dirs).parts[0] != Path(conf.SOUND_FILES_SOURCE).stem
+
+
+def check_no_subdir_crit(parent_dirs):
+    return len(list(Path(parent_dirs).parents)) == 1
+
+
+def check_top_dir_is_conf_top_dir():
+    return not Path(conf.SOUND_FILES_SOURCE).stem == conf.TOP_DIR_NAME
+
+
+def manage_dir_structure(file):
+    parent_dirs = get_directory_structure_relative_to_config_path(file)
+    parent_dirs = get_top_dir_name_if_only_one_parent_dir(file, parent_dirs)
+
+    bool_top_dir_crit = check_top_dir_crit(parent_dirs)
+    bool_no_subdir = check_no_subdir_crit(parent_dirs)
+    bool_top_dir_is_conf = check_top_dir_is_conf_top_dir()
+
+    if (bool_top_dir_crit and bool_no_subdir) and bool_top_dir_is_conf:
+        parent_dirs = Path(Path(conf.SOUND_FILES_SOURCE).stem).joinpath(
+            parent_dirs
+        )
+    return parent_dirs
+
+
+def get_top_dir(parent_dirs):
+    return str(parent_dirs).split("/")[0]
+
+
+def gen_annotations(
+    file,
+    model: tf.keras.Model,
+    mod_label: str,
+    timestamp_foldername: str,
+    **kwargs,
+):
+    """
+    Load audio file, instantiate model, use it to predict labels, fill a
+    dataframe with the predicted labels as well as necessary information to
+    import the annotations in a annotation program (like Raven). Finally the
+    annotations are saved as a single text file in directories corresponding
+    to the model checkpoint name within the generated annotations directory.
+
+    Parameters
+    ----------
+    file : str or pathlib.Path object
+        file path
+    model : tf.keras.Model
+        tensorflow model
+    mod_label : str
+        label to clarify which model was used
+    timestamp_foldername : str
+        date time string foldername corresponding to the time the annotations were
+        computed
+    """
+    parent_dirs = manage_dir_structure(file)
+
+    channel = get_channel(get_top_dir(parent_dirs))
+
+    audio = load_audio(file, channel)
+    if audio is None:
+        raise ImportError(
+            f"The audio file `{str(file)}` cannot be loaded. Check if file has "
+            "one of the supported endings "
+            "(wav, mp3, flac, etc.)) and is not empty."
+        )
+    audio_batches = batch_audio(audio)
+
+    annotation_df = create_annotation_df(audio_batches, model, **kwargs)
+
+    save_path = (
+        Path(conf.GEN_ANNOTS_DIR)
+        .joinpath(timestamp_foldername)
+        .joinpath(conf.THRESH_LABEL)
+        .joinpath(parent_dirs)
+    )
+    save_path.mkdir(exist_ok=True, parents=True)
+    annotation_df.to_csv(
+        save_path.joinpath(f"{file.stem}_annot_{mod_label}.txt"), sep="\t"
+    )
+
+    return annotation_df

acodet/global_config.py

@@ -0,0 +1,144 @@
+##############################################################################
+
+# THIS FILE IS ONLY MEANT TO BE EDITED IF YOU ARE SURE!
+
+# PROGRAM FAILURE IS LIKELY TO OCCUR IF YOU ARE UNSURE OF THE
+# CONSEQUENCES OF YOUR CHANGES.
+
+# IF YOU HAVE CHANGED VALUES AND ARE ENCOUNTERING ERRORS,
+# STASH YOUR CHANGES ('git stash' in a git bash console in acodet directory)
+# AND THEN PULL AGAIN ('git pull' in a git bash console in acodet directory)
+
+##############################################################################
+
+import json
+import streamlit as st
+
+if "session_started" in st.session_state:
+    session = {**st.session_state}
+else:
+    with open("acodet/src/tmp_session.json", "r") as f:
+        session = json.load(f)
+
+####################  AUDIO PROCESSING PARAMETERS  ###########################
+## GLOBAL AUDIO PROCESSING PARAMETERS
+SR = session["sample_rate"]
+## MEL-SPECTROGRAM PARAMETERS
+
+# FFT window length
+STFT_FRAME_LEN = session["stft_frame_len"]
+
+# number of time bins for mel spectrogram
+N_TIME_BINS = session["number_of_time_bins"]
+
+## CALCULATION OF CONTEXT WINDOW LENGTH
+# calculation of context window in seconds to fit stft frame length
+# and number of freq bins. From the set time length, the stft frame length
+# is subtracted, as it stays constant. The remainder gets used to calculate
+# the module of that value and the freuqency bins - 1 -> this then gives the
+# correct number of samples per context window excluding the stft length,
+# which is added subsequently.
+set_length_samples = session["context_window_in_seconds"] * SR
+set_length_without_stft_frame = set_length_samples - STFT_FRAME_LEN
+set_length_fixed = set_length_without_stft_frame % (N_TIME_BINS - 1)
+
+CONTEXT_WIN = int(
+    set_length_without_stft_frame - set_length_fixed + STFT_FRAME_LEN
+)
+
+CONTEXT_WIN_S_CORRECTED = CONTEXT_WIN / SR
+
+# downsample every audio file to this frame rate to ensure comparability
+DOWNSAMPLE_SR = False
+
+## Settings for Creation of Tfrecord Dataset
+# limit of context windows in a tfrecords file
+TFRECS_LIM = session["tfrecs_limit_per_file"]
+# train/test split
+TRAIN_RATIO = session["train_ratio"]
+# test/val split
+TEST_VAL_RATIO = session["test_val_ratio"]
+
+## Model Parameters
+# threshold for predictions
+THRESH = session["thresh"]
+# simple limit for hourly presence
+SIMPLE_LIMIT = session["simple_limit"]
+# sequence criterion threshold
+SEQUENCE_THRESH = session["sequence_thresh"]
+# sequence criterion limit
+SEQUENCE_LIMIT = session["sequence_limit"]
+# number of consecutive winodws for sequence criterion
+SEQUENCE_CON_WIN = session["sequence_con_win"]
+# limit for colorbar for hourly annotations
+HR_CNTS_VMAX = session["max_annots_per_hour"]
+# prediction window limit
+PRED_WIN_LIM = session["prediction_window_limit"]
+
+# calculated global variables
+FFT_HOP = (CONTEXT_WIN - STFT_FRAME_LEN) // (N_TIME_BINS - 1)
+PRED_BATCH_SIZE = PRED_WIN_LIM * CONTEXT_WIN
+
+## Paths
+TFREC_DESTINATION = session["tfrecords_destination_folder"]
+ANNOT_DEST = session["annotation_destination"]
+REV_ANNOT_SRC = session["reviewed_annotation_source"]
+GEN_ANNOT_SRC = session["generated_annotation_source"]
+SOUND_FILES_SOURCE = session["sound_files_source"]
+GEN_ANNOTS_DIR = session["generated_annotations_folder"]
+ANNOTS_TIMESTAMP_FOLDER = session["annots_timestamp_folder"]
+# model directory
+MODEL_DIR = "acodet/src/models"
+# model name
+MODEL_NAME = session["model_name"]
+TOP_DIR_NAME = session["top_dir_name"]
+THRESH_LABEL = session["thresh_label"]
+
+#############  ANNOTATIONS  #####################################
+DEFAULT_THRESH = session["default_threshold"]
+ANNOTATION_DF_FMIN = session["annotation_df_fmin"]
+ANNOTATION_DF_FMAX = session["annotation_df_fmax"]
+## Column Names
+# column name for annotation prediction values
+ANNOTATION_COLUMN = "Prediction/Comments"
+
+
+#################### RUN CONFIGURATION  ######################################
+RUN_CONFIG = session["run_config"]
+PRESET = session["predefined_settings"]
+
+#################### TRAINING CONFIG  ########################################
+
+MODELCLASSNAME = session["ModelClassName"]
+BATCH_SIZE = session["batch_size"]
+EPOCHS = session["epochs"]
+LOAD_CKPT_PATH = session["load_ckpt_path"]
+LOAD_G_CKPT = session["load_g_ckpt"]
+KERAS_MOD_NAME = session["keras_mod_name"]
+STEPS_PER_EPOCH = session["steps_per_epoch"]
+TIME_AUGS = session["time_augs"]
+MIXUP_AUGS = session["mixup_augs"]
+SPEC_AUG = session["spec_aug"]
+DATA_DESCRIPTION = session["data_description"]
+INIT_LR = float(session["init_lr"])
+FINAL_LR = float(session["final_lr"])
+PRE_BLOCKS = session["pre_blocks"]
+F_SCORE_BETA = session["f_score_beta"]
+F_SCORE_THRESH = session["f_score_thresh"]
+UNFREEZE = session["unfreeze"]
+
+
+##################### HOURLY PRESENCE DIR AND FILE NAMES #####################
+
+HR_CNTS_SL = "hourly_annotation_simple_limit"
+HR_PRS_SL = "hourly_presence_simple_limit"
+HR_CNTS_SC = "hourly_annotation_sequence_limit"
+HR_PRS_SC = "hourly_presence_sequence_limit"
+HR_VAL_PATH = session["hourly_presence_validation_path"]
+
+# column name for daily annotations (cumulative counts)
+HR_DA_COL = "daily_annotations"
+# column name for daily presence (binary)
+HR_DP_COL = "Daily_Presence"
+
+STREAMLIT = session["streamlit"]

acodet/hourly_presence.py

@@ -0,0 +1,765 @@
+import pandas as pd
+import numpy as np
+from acodet.funcs import get_files, get_dt_filename
+import acodet.global_config as conf
+from pathlib import Path
+import matplotlib.pyplot as plt
+import datetime as dt
+import seaborn as sns
+
+sns.set_theme()
+sns.set_style("white")
+
+
+def hourly_prs(df: pd.DataFrame, lim: int = 10):
+    """
+    Compute hourly presence.
+
+    Parameters
+    ----------
+    df : pd.DataFrame
+        dataframe containing annotations
+    lim : int, optional
+        limit for binary presence judgement, by default 10
+
+    Returns
+    -------
+    int
+        either 0 or 1 - 0 if less than lim annotations are present, 1 if more
+    """
+    if len(df) > lim:
+        return 1
+    else:
+        return 0
+
+
+def daily_prs(df: pd.DataFrame):
+    """
+    Compute daily presence. If at least one hour is present, the day is
+    considered present.
+
+    Parameters
+    ----------
+    df : pd.Dataframe
+        dataframe containing annotations
+
+    Returns
+    -------
+    int
+        0 or 1 - 0 if no hour is present, 1 if at least one hour is present
+    """
+    if 1 in df.loc[len(df), h_of_day_str()].values:
+        return 1
+    else:
+        return 0
+
+
+def get_val(path: str or Path):
+    """
+    Get validation dataframe.
+
+    Parameters
+    ----------
+    path : str or Path
+        path to validation dataframe
+
+    Returns
+    -------
+    pd.Dataframe
+        validation dataframe
+    """
+    return pd.read_csv(path)
+
+
+def h_of_day_str():
+    return ["%.2i:00" % i for i in np.arange(24)]
+
+
+def find_thresh05_path_in_dir(time_dir):
+    """
+    Get corrects paths leading to thresh_0.5 directory contatining annotations.
+    Correct for incorrect paths, that already contain the thresh_0.5 path.
+
+    Parameters
+    ----------
+    time_dir : str
+        if run.py is run directly, a path to a specific timestamp can be passed
+
+    Returns
+    -------
+    pathlib.Path
+        correct path leading to thresh_0.5 directory
+    """
+    root = Path(conf.GEN_ANNOT_SRC)
+    if root.parts[-1] == conf.THRESH_LABEL:
+        root = root.parent
+    elif root.parts[-1] == "thresh_0.9":
+        root = root.parent
+
+    if not time_dir:
+        if root.joinpath(conf.THRESH_LABEL).exists():
+            path = root.joinpath(conf.THRESH_LABEL)
+        else:
+            path = root
+    else:
+        path = (
+            Path(conf.GEN_ANNOTS_DIR)
+            .joinpath(time_dir)
+            .joinpath(conf.THRESH_LABEL)
+        )
+    return path
+
+
+def init_date_tuple(files):
+    dates = list(
+        map(lambda x: get_dt_filename(x.stem.split("_annot")[0]), files)
+    )
+    date_hour_tuple = list(
+        map(lambda x: (str(x.date()), "%.2i:00" % int(x.hour)), dates)
+    )
+
+    return np.unique(date_hour_tuple, axis=0, return_counts=True)
+
+
+def compute_hourly_pres(
+    time_dir=None,
+    thresh=conf.THRESH,
+    lim=conf.SIMPLE_LIMIT,
+    thresh_sc=conf.SEQUENCE_THRESH,
+    lim_sc=conf.SEQUENCE_LIMIT,
+    sc=False,
+    fetch_config_again=False,
+    **kwargs,
+):
+    if fetch_config_again:
+        import importlib
+
+        importlib.reload(conf)
+        thresh = conf.THRESH
+        lim = conf.SIMPLE_LIMIT
+        thresh_sc = conf.SEQUENCE_THRESH
+        lim_sc = conf.SEQUENCE_LIMIT
+
+    path = find_thresh05_path_in_dir(time_dir)
+
+    if "multi_datasets" in conf.session:
+        directories = [
+            [d for d in p.iterdir() if d.is_dir()]
+            for p in path.iterdir()
+            if p.is_dir()
+        ][0]
+    else:
+        directories = [p for p in path.iterdir() if p.is_dir()]
+    directories = [d for d in directories if not d.stem == "analysis"]
+
+    for ind, fold in enumerate(directories):
+        files = get_files(location=fold, search_str="**/*txt")
+        files.sort()
+
+        annots = return_hourly_pres_df(
+            files,
+            thresh,
+            thresh_sc,
+            lim,
+            lim_sc,
+            sc,
+            fold,
+            dir_ind=ind,
+            total_dirs=len(directories),
+            **kwargs,
+        )
+        if "save_filtered_selection_tables" in kwargs:
+            top_dir_path = path.parent.joinpath(conf.THRESH_LABEL).joinpath(
+                fold.stem
+            )
+        else:
+            top_dir_path = path.joinpath(fold.stem)
+
+        annots.df.to_csv(get_path(top_dir_path, conf.HR_PRS_SL))
+        annots.df_counts.to_csv(get_path(top_dir_path, conf.HR_CNTS_SL))
+        if not "dont_save_plot" in kwargs.keys():
+            for metric in (conf.HR_CNTS_SL, conf.HR_PRS_SL):
+                plot_hp(top_dir_path, lim, thresh, metric)
+
+        if sc:
+            annots.df_sc.to_csv(get_path(top_dir_path, conf.HR_PRS_SC))
+            annots.df_sc_cnt.to_csv(get_path(top_dir_path, conf.HR_CNTS_SC))
+            if not "dont_save_plot" in kwargs.keys():
+                for metric in (conf.HR_CNTS_SC, conf.HR_PRS_SC):
+                    plot_hp(top_dir_path, lim_sc, thresh_sc, metric)
+        print("\n")
+
+
+def get_end_of_last_annotation(annotations):
+    """
+    Get number of seconds from beginning to the end of the last annotation.
+
+    Parameters
+    ----------
+    annotations : pd.DataFrame
+        annotation dataframe
+
+    Returns
+    -------
+    int or bool
+        False or number of seconds until last annotation
+    """
+    if len(annotations) == 0:
+        return False
+    else:
+        return int(annotations["End Time (s)"].iloc[-1])
+
+
+def init_new_dt_if_exceeding_3600_s(h, date, hour):
+    """
+    Return new date and hour string if annotations exceed an hour. This
+    ensures that hour presence is still computed even if a recording
+    exceeds an hour.
+
+    Parameters
+    ----------
+    h : int
+        number of hours
+    date : str
+        date string
+    hour : str
+        hour string
+
+    Returns
+    -------
+    tuple
+        date and hour string
+    """
+    if h > 0:
+        new_dt = dt.datetime.strptime(
+            date + hour, "%Y-%m-%d%H:00"
+        ) + dt.timedelta(hours=1)
+        date = str(new_dt.date())
+        hour = "%.2i:00" % new_dt.hour
+    return date, hour
+
+
+class ProcessLimits:
+    def __init__(
+        self,
+        files,
+        thresh,
+        thresh_sc,
+        lim,
+        lim_sc,
+        sc,
+        dir_ind,
+        total_dirs,
+        return_counts,
+    ):
+        """
+        Handle processing of hourly and daily annotations counts and presence.
+        A class object is created and used to go through all files and calculate
+        the presence and annotation count metrics for a given hour within the
+        file. If more than one hour exists within a file, the metrics are counted
+        for every hour. If multiple files make up one hour they are concatenated
+        before processing. Simple limit and sequence limit processing is handled
+        by this class.
+
+        Parameters
+        ----------
+        files : list
+            pathlib.Path objects linking to file path
+        thresh : float
+            threshold
+        thresh_sc : float
+            threshold for sequence limit
+        lim : int
+            limit for simple limit presence
+        lim_sc : int
+            limit for sequence limit
+        sc : bool
+            sequence limit yes or no
+        dir_ind : int
+            directory index, in case multiple dirs are processed
+        total_dirs : int
+            number of total directories
+        return_counts : bool
+            return annotation counts or only binary presence
+        """
+        self.df = pd.DataFrame(
+            columns=["Date", conf.HR_DP_COL, *h_of_day_str()]
+        )
+        self.df_sc = self.df.copy()
+        self.df_counts = pd.DataFrame(
+            columns=["Date", conf.HR_DA_COL, *h_of_day_str()]
+        )
+        self.df_sc_cnt = self.df_counts.copy()
+        self.files = files
+        self.thresh = thresh
+        self.thresh_sc = thresh_sc
+        self.sc = sc
+        self.lim_sc = lim_sc
+        self.lim = lim
+        self.dir_ind = dir_ind
+        self.total_dirs = total_dirs
+        self.return_counts = return_counts
+
+        self.file_ind = 0
+        self.row = 0
+        self.n_prec_preds = conf.SEQUENCE_CON_WIN
+        self.n_exceed_thresh = 4
+
+    def concat_files_within_hour(self, count):
+        """
+        Concatenate files within one hour. Relevant if multiple files make
+        up one hour.
+
+        Parameters
+        ----------
+        count : int
+            number of files making up given hour
+        """
+        self.annot_all = pd.DataFrame()
+        self.filtered_annots = pd.DataFrame()
+        for _ in range(count):
+            self.annot_all = pd.concat(
+                [
+                    self.annot_all,
+                    pd.read_csv(self.files[self.file_ind], sep="\t"),
+                ]
+            )
+            self.file_ind += 1
+
+    def seq_crit(self, annot):
+        """
+        Sequence limit calculation. Initially all predictions are thresholded.
+        After that two boolean arrays are created. Using the AND operator for
+        these two arrays only returns the values that pass the sequence limit.
+        This means that within the number of consecutive windows
+        (self.n_prec_anns) more than the self.lim_sc number of windows have to
+        exceed the value of self.thresh_sc. Depending on the settings this
+        function is cancelled early if only binary presence is of interest.
+        A filtered annotations dataframe is saved that only has the predictions
+        that passed the filtering process.
+
+        Parameters
+        ----------
+        annot : pandas.DataFrame
+            annotations of the given hour
+
+        Returns
+        -------
+        int
+            either 1 if only binary presence is relevant or the number of
+            annotations that passed the sequence limit in the given hour
+        """
+        sequ_crit = 0
+        annot = annot.loc[annot[conf.ANNOTATION_COLUMN] >= self.thresh_sc]
+        for i, row in annot.iterrows():
+            bool1 = 0 < (row["Begin Time (s)"] - annot["Begin Time (s)"])
+            bool2 = (
+                row["Begin Time (s)"] - annot["Begin Time (s)"]
+            ) < self.n_prec_preds * conf.CONTEXT_WIN / conf.SR
+            self.prec_anns = annot.loc[bool1 * bool2]
+            if len(self.prec_anns) > self.n_exceed_thresh:
+                sequ_crit += 1
+                self.filtered_annots = pd.concat(
+                    [self.filtered_annots, self.prec_anns.iloc[-1:]]
+                )
+                # this stops the function as soon as the limit is met once
+                if not self.return_counts:
+                    return 1
+        return sequ_crit
+
+    def get_end_of_last_annotation(self):
+        """
+        Get the time corresponding to the last annotation in current file.
+        """
+        if len(self.annot_all) == 0:
+            self.end = False
+        else:
+            self.end = int(self.annot_all["End Time (s)"].iloc[-1])
+
+    def filter_files_of_hour_by_limit(self, date, hour):
+        """
+        Process the annotation counts and binary presence for a given
+        hour in the dataset. This function is quite cryptic because there
+        are several dataframes that have to be updated to insert the
+        correct number of annotations and the binary presence value
+        for the given hour and date.
+
+        Parameters
+        ----------
+        date : string
+            date string
+        hour : string
+            hour string
+        """
+        for h in range(0, self.end or 1, 3600):
+            fil_h_ann = self.annot_all.loc[
+                (h < self.annot_all["Begin Time (s)"])
+                & (self.annot_all["Begin Time (s)"] < h + 3600)
+            ]
+            date, hour = init_new_dt_if_exceeding_3600_s(h, date, hour)
+
+            fil_h_ann = fil_h_ann.loc[
+                fil_h_ann[conf.ANNOTATION_COLUMN] >= self.thresh
+            ]
+            if not date in self.df["Date"].values:
+                if not self.row == 0:
+                    self.df.loc[self.row, conf.HR_DP_COL] = daily_prs(self.df)
+                    self.df_counts.loc[self.row, conf.HR_DA_COL] = sum(
+                        self.df_counts.loc[
+                            len(self.df_counts), h_of_day_str()
+                        ].values
+                    )
+
+                    if self.sc:
+                        self.df_sc.loc[self.row, conf.HR_DP_COL] = daily_prs(
+                            self.df_sc
+                        )
+                        self.df_sc_cnt.loc[self.row, conf.HR_DA_COL] = sum(
+                            self.df_sc_cnt.loc[
+                                len(self.df_sc_cnt), h_of_day_str()
+                            ].values
+                        )
+
+                self.row += 1
+                self.df.loc[self.row, "Date"] = date
+                self.df_counts.loc[self.row, "Date"] = date
+                if self.sc:
+                    self.df_sc.loc[self.row, "Date"] = date
+                    self.df_sc_cnt.loc[self.row, "Date"] = date
+
+            self.df.loc[self.row, hour] = hourly_prs(fil_h_ann, lim=self.lim)
+            self.df_counts.loc[self.row, hour] = len(fil_h_ann)
+
+            if self.file_ind == len(self.files):
+                self.df.loc[self.row, conf.HR_DP_COL] = daily_prs(self.df)
+                self.df_counts.loc[self.row, conf.HR_DA_COL] = sum(
+                    self.df_counts.loc[
+                        len(self.df_counts), h_of_day_str()
+                    ].values
+                )
+
+                if self.sc:
+                    self.df_sc.loc[self.row, conf.HR_DP_COL] = daily_prs(
+                        self.df_sc
+                    )
+                    self.df_sc_cnt.loc[self.row, conf.HR_DA_COL] = sum(
+                        self.df_sc_cnt.loc[
+                            len(self.df_sc_cnt), h_of_day_str()
+                        ].values
+                    )
+
+            if self.sc:
+                self.df_sc_cnt.loc[self.row, hour] = self.seq_crit(fil_h_ann)
+                self.df_sc.loc[self.row, hour] = int(
+                    bool(self.df_sc_cnt.loc[self.row, hour])
+                )
+
+    def save_filtered_selection_tables(self, dataset_path):
+        """
+        Save the selection tables under a new directory with the
+        chosen filter settings. Depending if sequence limit is chosen
+        or not a directory name is chosen and saved in the parent
+        timestamp foldername of the current inference session.
+
+        Parameters
+        ----------
+        dataset_path : pathlib.Path
+            path to dataset in current annotation timestamp folder
+        """
+        if self.sc:
+            thresh_label = f"thresh_{self.thresh_sc}_seq_{self.lim_sc}"
+        else:
+            thresh_label = f"thresh_{self.thresh}_sim"
+        conf.THRESH_LABEL = thresh_label
+        new_thresh_path = Path(conf.GEN_ANNOT_SRC).joinpath(thresh_label)
+        new_thresh_path = new_thresh_path.joinpath(
+            self.files[self.file_ind - 1]
+            .relative_to(dataset_path.parent)
+            .parent
+        )
+        new_thresh_path.mkdir(exist_ok=True, parents=True)
+        file_path = new_thresh_path.joinpath(
+            self.files[self.file_ind - 1].stem
+            + self.files[self.file_ind - 1].suffix
+        )
+        if not self.sc:
+            self.filtered_annots = self.annot_all.loc[
+                self.annot_all[conf.ANNOTATION_COLUMN] >= self.thresh
+            ]
+        if len(self.filtered_annots) > 0:
+            self.filtered_annots.index = self.filtered_annots.Selection
+            self.filtered_annots.pop("Selection")
+            self.filtered_annots.to_csv(file_path, sep="\t")
+
+    def update_annotation_progbar(self, **kwargs):
+        """
+        Update the annotation progbar in the corresponding streamlit widget.
+        """
+        import streamlit as st
+
+        inner_counter = self.file_ind / len(self.files)
+        outer_couter = self.dir_ind / self.total_dirs
+        counter = inner_counter * 1 / self.total_dirs + outer_couter
+
+        if "preset" in kwargs:
+            st.session_state.progbar_update.progress(
+                counter,
+                text="Progress",
+            )
+            if counter == 1 and "update_plot" in kwargs:
+                st.write("Plot updated")
+                st.button("Update plot")
+        elif conf.PRESET == 3:
+            kwargs["progbar1"].progress(
+                counter,
+                text="Progress",
+            )
+
+
+def return_hourly_pres_df(
+    files,
+    thresh,
+    thresh_sc,
+    lim,
+    lim_sc,
+    sc,
+    path,
+    total_dirs,
+    dir_ind,
+    return_counts=True,
+    save_filtered_selection_tables=False,
+    **kwargs,
+):
+    """
+    Return the hourly presence and hourly annotation counts for all files
+    within the chosen dataset. Processing is handled by the ProcessLimits class
+    this is the caller function.
+
+    Parameters
+    ----------
+    files : list
+        pathlib.Path objects linking to files
+    thresh : float
+        threshold
+    thresh_sc : float
+        threshold for sequence limit
+    lim : int
+        limit of simple limit for binary presence
+    lim_sc : int
+        limit for sequence limit
+    sc : bool
+        sequence limit yes or no
+    path : pathlib.Path
+        path to current dataset in annotations folder
+    total_dirs : int
+        number of directories to be annotated
+    dir_ind : int
+        index of directory
+    return_counts : bool, optional
+        only binary presence or hourly counts, by default True
+    save_filtered_selection_tables : bool, optional
+        whether to save the filtered selection tables or not, by default False
+
+    Returns
+    -------
+    ProcessLimits object
+        contains all dataframes with the hourly and daily metrics
+    """
+    if not isinstance(path, Path):
+        path = Path(path)
+
+    tup, counts = init_date_tuple(files)
+    filt_annots = ProcessLimits(
+        files,
+        thresh,
+        thresh_sc,
+        lim,
+        lim_sc,
+        sc,
+        dir_ind,
+        total_dirs,
+        return_counts,
+    )
+    for (date, hour), count in zip(tup, counts):
+        filt_annots.concat_files_within_hour(count)
+
+        filt_annots.get_end_of_last_annotation()
+
+        filt_annots.filter_files_of_hour_by_limit(date, hour)
+
+        if save_filtered_selection_tables:
+            filt_annots.save_filtered_selection_tables(path)
+
+        print(
+            f"Computing files in {path.stem}: "
+            f"{filt_annots.file_ind}/{len(files)}",
+            end="\r",
+        )
+        if "preset" in kwargs or conf.PRESET == 3 and conf.STREAMLIT:
+            filt_annots.update_annotation_progbar(**kwargs)
+
+    return filt_annots
+
+
+def get_path(path, metric):
+    if not path.stem == "analysis":
+        save_path = path.parent.joinpath("analysis").joinpath(path.stem)
+    else:
+        save_path = path
+    save_path.mkdir(exist_ok=True, parents=True)
+    return save_path.joinpath(f"{metric}.csv")
+
+
+def get_title(metric):
+    if "annotation" in metric:
+        return "Annotation counts for each hour"
+    elif "presence" in metric:
+        return "Hourly presence"
+
+
+def plot_hp(path, lim, thresh, metric):
+    path = get_path(path, metric)
+    df = pd.read_csv(path)
+    h_pres = df.loc[:, h_of_day_str()]
+    h_pres.index = df["Date"]
+    plt.figure(figsize=[8, 6])
+    plt.title(
+        f"{get_title(metric)}, limit={lim:.0f}, " f"threshold={thresh:.2f}"
+    )
+    if "presence" in metric:
+        d = {"vmin": 0, "vmax": 1}
+    else:
+        d = {"vmax": conf.HR_CNTS_VMAX}
+    sns.heatmap(h_pres.T, cmap="crest", **d)
+    plt.ylabel("hour of day")
+    plt.tight_layout()
+    plt.savefig(
+        path.parent.joinpath(f"{metric}_{thresh:.2f}_{lim:.0f}.png"), dpi=150
+    )
+    plt.close()
+
+
+def calc_val_diff(
+    time_dir=None,
+    thresh=conf.THRESH,
+    lim=conf.SIMPLE_LIMIT,
+    thresh_sc=conf.SEQUENCE_THRESH,
+    lim_sc=conf.SEQUENCE_LIMIT,
+    sc=True,
+    **kwargs,
+):
+    path = find_thresh05_path_in_dir(time_dir)
+    for ind, fold in enumerate(path.iterdir()):
+        if not fold.joinpath("analysis").joinpath(conf.HR_VAL_PATH).exists():
+            continue
+
+        df_val = get_val(fold.joinpath("analysis").joinpath(conf.HR_VAL_PATH))
+        hours_of_day = ["%.2i:00" % i for i in np.arange(24)]
+        files = get_files(
+            location=path.joinpath(fold.stem), search_str="**/*txt"
+        )
+        files.sort()
+
+        annots = return_hourly_pres_df(
+            files,
+            thresh,
+            thresh_sc,
+            lim,
+            lim_sc,
+            sc,
+            fold,
+            total_dirs=len(list(path.iterdir())),
+            dir_ind=ind + 1,
+            return_counts=False,
+            **kwargs,
+        )
+
+        d, incorrect, df_diff = dict(), dict(), dict()
+        for agg_met, df_metric in zip(("sl", "sq"), (annots.df, annots.df_sc)):
+            df_val.index = df_metric.index
+            df_diff.update(
+                {
+                    agg_met: df_val.loc[:, hours_of_day]
+                    - df_metric.loc[:, hours_of_day]
+                }
+            )
+
+            results = np.unique(df_diff[agg_met])
+            d.update(
+                {agg_met: dict({"true": 0, "false_pos": 0, "false_neg": 0})}
+            )
+            for met, val in zip(d[agg_met].keys(), (0, -1, 1)):
+                if val in results:
+                    d[agg_met][met] = len(np.where(df_diff[agg_met] == val)[0])
+            incorrect.update(
+                {agg_met: d[agg_met]["false_pos"] + d[agg_met]["false_neg"]}
+            )
+        perf_df = pd.DataFrame(d)
+
+        print(
+            "\n",
+            "l:",
+            lim,
+            "th:",
+            thresh,
+            "incorrect:",
+            incorrect["sl"],
+            "%.2f" % (incorrect["sl"] / (len(df_diff["sl"]) * 24) * 100),
+        )
+        print(
+            "l:",
+            lim_sc,
+            "th:",
+            thresh_sc,
+            "sc_incorrect:",
+            incorrect["sq"],
+            "%.2f" % (incorrect["sq"] / (len(df_diff["sl"]) * 24) * 100),
+        )
+
+        annots.df.to_csv(
+            Path(fold)
+            .joinpath("analysis")
+            .joinpath(f"th{thresh}_l{lim}_hourly_presence.csv")
+        )
+        annots.df_sc.to_csv(
+            Path(fold)
+            .joinpath("analysis")
+            .joinpath(f"th{thresh_sc}_l{lim_sc}_hourly_pres_sequ_crit.csv")
+        )
+        df_diff["sl"].to_csv(
+            Path(fold)
+            .joinpath("analysis")
+            .joinpath(f"th{thresh}_l{lim}_diff_hourly_presence.csv")
+        )
+        df_diff["sq"].to_csv(
+            Path(fold)
+            .joinpath("analysis")
+            .joinpath(
+                f"th{thresh_sc}_l{lim_sc}_diff_hourly_pres_sequ_crit.csv"
+            )
+        )
+        perf_df.to_csv(
+            Path(fold)
+            .joinpath("analysis")
+            .joinpath(f"th{thresh_sc}_l{lim_sc}_diff_performance.csv")
+        )
+
+
+def plot_varying_limits(annotations_path=conf.ANNOT_DEST):
+    thresh_sl, thresh_sc = 0.9, 0.9
+    for lim_sl, lim_sc in zip(np.linspace(10, 48, 20), np.linspace(1, 20, 20)):
+        for lim, thresh in zip((lim_sl, thresh_sl), (lim_sc, thresh_sc)):
+            compute_hourly_pres(
+                annotations_path,
+                thresh_sc=thresh_sc,
+                lim_sc=lim_sc,
+                thresh=thresh,
+                lim=lim,
+            )
+            for metric in (
+                conf.HR_CNTS_SC,
+                conf.HR_CNTS_SL,
+                conf.HR_PRS_SC,
+                conf.HR_PRS_SL,
+            ):
+                plot_hp(annotations_path, lim, thresh, metric)

acodet/humpback_model_dir/README.md

@@ -0,0 +1,15 @@
+# Humpback-specific model Python code
+
+This is not part of the multispecies\_whale\_detection package proper but rather
+related prior work being made available for reference to users who want to make
+fine-grained modifications using the weights from the SavedModel released at
+
+https://tfhub.dev/google/humpback\_whale/1
+
+The best way to learn details is to read the comments in the Python source
+files. Very basic usage:
+
+```
+import humpback_model
+
+model = humpback_model.Model.load_from_tf_hub()

acodet/humpback_model_dir/front_end.py

@@ -0,0 +1,105 @@
+# Copyright 2022 Google LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Functions for representing raw audio as images.
+
+Having these in the graph allows hyperparameters involved in this representation
+to be fixed at training time instead of needing to materialize extracted
+features on disk.
+"""
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import collections
+
+import tensorflow as tf
+from acodet import global_config as conf
+
+Config = collections.namedtuple(
+    "Config",
+    [
+        "stft_frame_length",
+        "stft_frame_step",
+        "freq_bins",
+        "sample_rate",
+        "lower_f",
+        "upper_f",
+    ],
+)
+"""Configuration for the front end.
+
+Attributes:
+  stft_frame_length: The window length for the STFT in samples.
+  stft_frame_step: The number of samples from the start of one STFT snapshot to
+    the next.
+  freq_bins: The number of mel bins in the spectrogram.
+  lower_f: Lower boundary of mel bins in Hz.
+  upper_f: Upper boundary of mel bins in Hz.
+"""
+
+Config.__new__.__defaults__ = (
+    conf.STFT_FRAME_LEN,
+    conf.FFT_HOP,
+    64,
+    conf.SR,
+    0.0,
+    conf.SR / 2,
+)
+
+
+class MelSpectrogram(tf.keras.layers.Layer):
+    """Keras layer that converts a waveform to an amplitude mel spectrogram."""
+
+    def __init__(self, config=None, name="mel_spectrogram"):
+        super(MelSpectrogram, self).__init__(name=name)
+        if config is None:
+            config = Config()
+        self.config = config
+
+    def get_config(self):
+        config = super().get_config()
+        config.update({key: val for key, val in config.items()})
+        return config
+
+    def build(self, input_shape):
+        self._stft = tf.keras.layers.Lambda(
+            lambda t: tf.signal.stft(
+                tf.squeeze(t, 2),
+                frame_length=self.config.stft_frame_length,
+                frame_step=self.config.stft_frame_step,
+            ),
+            name="stft",
+        )
+        num_spectrogram_bins = self._stft.compute_output_shape(input_shape)[-1]
+        self._bin = tf.keras.layers.Lambda(
+            lambda t: tf.square(
+                tf.tensordot(
+                    tf.abs(t),
+                    tf.signal.linear_to_mel_weight_matrix(
+                        num_mel_bins=self.config.freq_bins,
+                        num_spectrogram_bins=num_spectrogram_bins,
+                        sample_rate=self.config.sample_rate,
+                        lower_edge_hertz=self.config.lower_f,
+                        upper_edge_hertz=self.config.upper_f,
+                        name="matrix",
+                    ),
+                    1,
+                )
+            ),
+            name="mel_bins",
+        )
+
+    def call(self, inputs):
+        return self._bin(self._stft(inputs))

acodet/humpback_model_dir/humpback_model.py

@@ -0,0 +1,384 @@
+# Copyright 2022 Google LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Keras implementation of the NOAA PIFSC humpback whale detection model.
+
+The released SavedModel is available at
+
+https://tfhub.dev/google/humpback_whale/1
+
+Model.load_from_tf_hub is a convenience method that downloads the SavedModel and
+initializes the Model defined here with the weights from the SavedModel.
+
+To save redownloading, assuming a local copy is in model_dir, use:
+
+model = humpback_model.Model()
+model.load_weights(model_dir)
+
+This is the Python code that was used to export that model. We release it to
+enable users who want to do finer-grained fine-tuning than what is possible
+with the SavedModel alone, for example freezing or removing layers.
+
+The code implements a standard ResNet50 in Keras. Although off-the-shelf
+implementations exist, this reimplementation was necessary to have exact control
+over variable names, to load weights from a model that had been trained in TF1.
+"""
+
+import shutil
+import tarfile
+import tempfile
+import time
+from urllib import request
+
+import tensorflow as tf
+
+from . import front_end
+from . import leaf_pcen
+
+TF_HUB_URL = (
+    "https://tfhub.dev/google/humpback_whale/1?tf-hub-format=compressed"
+)
+NUM_CLASSES = 1
+NUM_AUDIO_CHANNELS = 1
+CONTEXT_WAVEFORM_SHAPE = [None, 39124, NUM_AUDIO_CHANNELS]
+CONTEXT_SPECTROGRAM_SHAPE = [None, 128, 64]
+EMBEDDING_DIMENSION = 2048
+
+
+def BatchNormalization(name=None):
+    """Defaults some of the arguments of Keras' BatchNormalization layer."""
+    return tf.keras.layers.BatchNormalization(
+        epsilon=1e-4,
+        momentum=0.9997,
+        scale=False,
+        center=True,
+        name=name,
+    )
+
+
+def Conv2D(filters, kernel_size, strides=(1, 1), padding="VALID", name=None):
+    """Defaults some of the arguments of Keras' Conv2D layer."""
+    return tf.keras.layers.Conv2D(
+        filters,
+        kernel_size,
+        strides,
+        padding=padding,
+        activation=None,
+        use_bias=False,
+        name=name,
+    )
+
+
+def _call_layers(layers, inputs):
+    """Applies the function composition of layers to inputs, like Sequential."""
+    t = inputs
+    for layer in layers:
+        t = layer(t)
+    return t
+
+
+class ResidualPath(tf.keras.layers.Layer):
+    """Layer for the residual "skip" connection in a ResNet block."""
+
+    def __init__(self, num_output_channels, input_stride):
+        super(ResidualPath, self).__init__(name="residual_path")
+        self.num_output_channels = num_output_channels
+        self.input_stride = input_stride
+
+    def build(self, input_shape):
+        num_input_channels = input_shape[-1]
+        if num_input_channels != self.num_output_channels:
+            self._layers = [
+                Conv2D(
+                    self.num_output_channels,
+                    kernel_size=1,
+                    strides=self.input_stride,
+                    padding="VALID",
+                    name="conv_residual",
+                ),
+                BatchNormalization(name="batch_normalization_residual"),
+            ]
+        else:
+            self._layers = []
+
+    def call(self, inputs):
+        return _call_layers(self._layers, inputs)
+
+
+class MainPath(tf.keras.layers.Layer):
+    """Layer for the bottleneck-and-convolutional "core" of a ResNet block."""
+
+    def __init__(self, num_inner_channels, num_output_channels, input_stride):
+        super(MainPath, self).__init__(name="main_path")
+        self.num_inner_channels = num_inner_channels
+        self.num_output_channels = num_output_channels
+        self.input_stride = input_stride
+
+    def build(self, input_shape):
+        num_input_channels = input_shape[-1]
+        if num_input_channels != self.num_output_channels:
+            bottleneck_padding = "VALID"
+        else:
+            bottleneck_padding = "SAME"
+        self._layers = [
+            Conv2D(
+                self.num_inner_channels,
+                kernel_size=1,
+                strides=self.input_stride,
+                padding=bottleneck_padding,
+                name="conv_bottleneck",
+            ),
+            BatchNormalization(name="batch_normalization_bottleneck"),
+            tf.keras.layers.ReLU(name="relu_bottleneck"),
+            Conv2D(
+                self.num_inner_channels,
+                kernel_size=3,
+                strides=1,
+                padding="SAME",
+                name="conv",
+            ),
+            BatchNormalization(name="batch_normalization"),
+            tf.keras.layers.ReLU(name="relu"),
+            Conv2D(
+                self.num_output_channels,
+                kernel_size=1,
+                strides=1,
+                padding="SAME",
+                name="conv_output",
+            ),
+            BatchNormalization(name="batch_normalization_output"),
+        ]
+
+    def call(self, inputs):
+        return _call_layers(self._layers, inputs)
+
+
+class Block(tf.keras.layers.Layer):
+    """Layer for a ResNet block."""
+
+    def __init__(
+        self,
+        num_inner_channels,
+        num_output_channels,
+        input_stride=1,
+        name="block",
+    ):
+        super(Block, self).__init__(name=name)
+        self.num_inner_channels = num_inner_channels
+        self.num_output_channels = num_output_channels
+        self.input_stride = input_stride
+
+    def build(self, input_shape):
+        self._residual_path = ResidualPath(
+            self.num_output_channels, self.input_stride
+        )
+        self._main_path = MainPath(
+            self.num_inner_channels,
+            self.num_output_channels,
+            self.input_stride,
+        )
+        self._activation = tf.keras.layers.ReLU(name="relu_output")
+
+    def call(self, features):
+        return self._activation(
+            self._residual_path(features) + self._main_path(features)
+        )
+
+
+class Group(tf.keras.layers.Layer):
+    """Layer for a group of ResNet blocks with common inner and outer depths."""
+
+    def __init__(
+        self, repeats, inner_channels, output_channels, input_stride, name
+    ):
+        super(Group, self).__init__(name=name)
+        assert repeats >= 1
+        self.repeats = repeats
+        self.inner_channels = inner_channels
+        self.output_channels = output_channels
+        self.input_stride = input_stride
+
+    def build(self, input_shape):
+        self._layers = [
+            Block(
+                self.inner_channels,
+                self.output_channels,
+                input_stride=self.input_stride,
+                name="block_widen",
+            )
+        ]
+        for i in range(1, self.repeats):
+            self._layers.append(
+                Block(
+                    self.inner_channels,
+                    self.output_channels,
+                    input_stride=1,
+                    name=("block_" + str(i)),
+                )
+            )
+
+    def call(self, inputs):
+        return _call_layers(self._layers, inputs)
+
+
+class PreBlocks(tf.keras.layers.Layer):
+    """Layer with first-layer convolutional filters."""
+
+    def build(self, input_shape):
+        self._layers = [
+            tf.keras.layers.Lambda(
+                lambda t: tf.expand_dims(t, 3), name="make_depth_one"
+            ),
+            Conv2D(64, 7, padding="SAME", name="conv"),
+            BatchNormalization(name="batch_normalization"),
+            tf.keras.layers.ReLU(name="relu"),
+            tf.keras.layers.MaxPool2D(3, 2, padding="SAME", name="pool"),
+        ]
+
+    def call(self, inputs):
+        return _call_layers(self._layers, inputs)
+
+
+class Embed(tf.keras.layers.Layer):
+    """Composition of layers transforming sectrogram to embedding vector.
+
+    The spectrogram has [128, 64] [time, frequency] bins. When the weights from TF
+    Hub are used, frequency whould be on a mel scale and PCEN applied, as in the
+    implementation of Model.__init__, later.
+    """
+
+    def build(self, input_shape):
+        self._layers = [
+            tf.keras.layers.InputLayer(input_shape=[128, 64]),
+            PreBlocks(),
+            Group(3, 64, 256, input_stride=1, name="group"),
+            Group(4, 128, 512, input_stride=2, name="group_1"),
+            Group(6, 256, 1024, input_stride=2, name="group_2"),
+            Group(3, 512, 2048, input_stride=2, name="group_3"),
+            tf.keras.layers.GlobalAveragePooling2D(name="pool"),
+        ]
+
+    def call(self, inputs):
+        return _call_layers(self._layers, inputs)
+
+
+class Model(tf.keras.Sequential):
+    """Full humpback detection Keras model with supplementary signatures.
+
+    See "Advanced Usage" on https://tfhub.dev/google/humpback_whale/1 for details
+    on the reusable SavedModels attributes (front_end, features, logits).
+
+    The "score" method is provided for variable-length inputs.
+    """
+
+    @classmethod
+    def load_from_tf_hub(cls):
+        """Downloads the SavedModel and initialized this class using its weights."""
+        with tempfile.NamedTemporaryFile() as temp_file:
+            with request.urlopen(TF_HUB_URL) as response:
+                shutil.copyfileobj(response, temp_file)
+            temp_file.flush()
+            temp_file.seek(0)
+            with tempfile.TemporaryDirectory() as temp_dir:
+                with tarfile.open(temp_file.name, "r:gz") as tar:
+                    tar.extractall(path=temp_dir)
+                model = cls()
+                model.load_weights(temp_dir)
+        return model
+
+    def __init__(self):
+        super(Model, self).__init__(
+            layers=[
+                front_end.MelSpectrogram(),
+                leaf_pcen.PCEN(
+                    alpha=0.98,
+                    delta=2.0,
+                    root=2.0,
+                    smooth_coef=0.025,
+                    floor=1e-6,
+                    trainable=True,
+                    name="pcen",
+                ),
+                Embed(),
+                tf.keras.layers.Dense(NUM_CLASSES),
+            ]
+        )
+        front_end_layers = self.layers[:2]
+        self._spectrogram, self._pcen = front_end_layers
+
+        # Parts exposed through Reusable SavedModels interface.
+        self.front_end = tf.keras.Sequential(
+            [tf.keras.layers.InputLayer([None, 1])] + front_end_layers
+        )
+        self.features = tf.keras.Sequential(
+            [tf.keras.layers.InputLayer([128, 64]), self.layers[2]]
+        )
+        self.logits = tf.keras.Sequential(
+            [tf.keras.layers.InputLayer([128, 64])] + self.layers[2:]
+        )
+
+    @tf.function(
+        input_signature=[
+            tf.TensorSpec(shape=(None, None, 1), dtype=tf.float32),
+            tf.TensorSpec(shape=tuple(), dtype=tf.int64),
+        ]
+    )
+    def score(self, waveform, context_step_samples):
+        """Scores each context window in an arbitrary-length waveform.
+
+        This is the clip-level version of __call__. It slices out short waveform
+        context windows of the duration expected by __call__, scores them as a
+        batch, and returns the corresponding scores.
+
+        Args:
+          waveform: [batch, samples, channels == 1] Tensor of PCM audio.
+          context_step_samples: Difference between the starts of two consecutive
+            context windows, in samples.
+
+        Returns:
+          Dict {'scores': [batch, num_windows, 1]} Tensor of per-context-window
+          model outputs. (Post-sigmoid, in [0, 1].)
+        """
+        batch_size = tf.shape(waveform)[0]
+        stft_frame_step_samples = 300
+        context_step_frames = tf.cast(
+            context_step_samples // stft_frame_step_samples, tf.dtypes.int32
+        )
+        mel_spectrogram = self._spectrogram(waveform)
+        context_duration_frames = self.features.input_shape[1]
+        context_windows = tf.signal.frame(
+            mel_spectrogram,
+            context_duration_frames,
+            context_step_frames,
+            axis=1,
+        )
+        num_windows = tf.shape(context_windows)[1]
+        windows_in_batch = tf.reshape(
+            context_windows, (-1,) + self.features.input_shape[1:]
+        )
+        per_window_pcen = self._pcen(windows_in_batch)
+        scores = tf.nn.sigmoid(self.logits(per_window_pcen))
+        return {"scores": tf.reshape(scores, [batch_size, num_windows, 1])}
+
+    @tf.function(input_signature=[])
+    def metadata(self):
+        config = self._spectrogram.config
+        return {
+            "input_sample_rate": tf.cast(config.sample_rate, tf.int64),
+            "context_width_samples": tf.cast(
+                config.stft_frame_step * (self.features.input_shape[1] - 1)
+                + config.stft_frame_length,
+                tf.int64,
+            ),
+            "class_names": tf.constant(["Mn"]),
+        }

acodet/humpback_model_dir/leaf_pcen.py

@@ -0,0 +1,103 @@
+# Copyright 2022 Google LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""PCEN implementation, forked from google-research/leaf-audio."""
+
+import tensorflow as tf
+
+
+class PCEN(tf.keras.layers.Layer):
+    """Per-Channel Energy Normalization.
+
+    This applies a fixed or learnable normalization by an exponential moving
+    average smoother, and a compression.
+    This implementation replicates the computation of fixed_pcen and
+    trainable_pcen defined in
+    google3/audio/hearing/tts/tensorflow/python/ops/pcen_ops.py.
+    See https://arxiv.org/abs/1607.05666 for more details.
+    """
+
+    def __init__(
+        self,
+        alpha: float,
+        smooth_coef: float,
+        delta: float = 2.0,
+        root: float = 2.0,
+        floor: float = 1e-6,
+        trainable: bool = False,
+        name="PCEN",
+    ):
+        """PCEN constructor.
+
+        Args:
+          alpha: float, exponent of EMA smoother
+          smooth_coef: float, smoothing coefficient of EMA
+          delta: float, bias added before compression
+          root: float, one over exponent applied for compression (r in the paper)
+          floor: float, offset added to EMA smoother
+          trainable: bool, False means fixed_pcen, True is trainable_pcen
+          name: str, name of the layer
+        """
+        super().__init__(name=name)
+        self._alpha_init = alpha
+        self._delta_init = delta
+        self._root_init = root
+        self._smooth_coef = smooth_coef
+        self._floor = floor
+        self._trainable = trainable
+
+    def build(self, input_shape):
+        num_channels = input_shape[-1]
+        self.alpha = self.add_weight(
+            name="alpha",
+            shape=[num_channels],
+            initializer=tf.keras.initializers.Constant(self._alpha_init),
+            trainable=self._trainable,
+        )
+        self.delta = self.add_weight(
+            name="delta",
+            shape=[num_channels],
+            initializer=tf.keras.initializers.Constant(self._delta_init),
+            trainable=self._trainable,
+        )
+        self.root = self.add_weight(
+            name="root",
+            shape=[num_channels],
+            initializer=tf.keras.initializers.Constant(self._root_init),
+            trainable=self._trainable,
+        )
+        self.ema = tf.keras.layers.SimpleRNN(
+            units=num_channels,
+            activation=None,
+            use_bias=False,
+            kernel_initializer=tf.keras.initializers.Identity(
+                gain=self._smooth_coef
+            ),
+            recurrent_initializer=tf.keras.initializers.Identity(
+                gain=1.0 - self._smooth_coef
+            ),
+            return_sequences=True,
+            trainable=False,
+        )
+
+    def call(self, inputs):
+        alpha = tf.math.minimum(self.alpha, 1.0)
+        root = tf.math.maximum(self.root, 1.0)
+        ema_smoother = self.ema(
+            inputs, initial_state=tf.gather(inputs, 0, axis=1)
+        )
+        one_over_root = 1.0 / root
+        output = (
+            inputs / (self._floor + ema_smoother) ** alpha + self.delta
+        ) ** one_over_root - self.delta**one_over_root
+        return output

acodet/models.py

@@ -0,0 +1,318 @@
+import tensorflow as tf
+from tensorflow_addons import metrics
+from pathlib import Path
+import zipfile
+import sys
+
+from acodet.funcs import get_val_labels
+from . import global_config as conf
+from .humpback_model_dir import humpback_model
+from .humpback_model_dir import front_end
+from .humpback_model_dir import leaf_pcen
+
+
+class ModelHelper:
+    """
+    Helper class to provide checkpoint loading and changing of input shape.
+    """
+
+    def load_ckpt(self, ckpt_path, ckpt_name="last"):
+        if isinstance(ckpt_path, Path):
+            ckpt_path = ckpt_path.stem
+        ckpt_path = (
+            Path("../trainings").joinpath(ckpt_path).joinpath("unfreeze_no-TF")
+        )  # TODO namen ändern
+        try:
+            file_path = ckpt_path.joinpath(f"cp-{ckpt_name}.ckpt.index")
+            if not file_path.exists():
+                ckpts = list(ckpt_path.glob("cp-*.ckpt*"))
+                ckpts.sort()
+                ckpt = ckpts[-1]
+            else:
+                ckpt = file_path
+            self.model.load_weights(
+                str(ckpt).replace(".index", "")
+            ).expect_partial()
+        except Exception as e:
+            print("Checkpoint not found.", e)
+
+    def change_input_to_array(self):
+        """
+        change input layers of model after loading checkpoint so that a file
+        can be predicted based on arrays rather than spectrograms, i.e.
+        reintegrate the spectrogram creation into the model.
+
+        Args:
+            model (tf.keras.Sequential): keras model
+
+        Returns:
+            tf.keras.Sequential: model with new arrays as inputs
+        """
+        model_list = self.model.layers
+        model_list.insert(0, tf.keras.layers.Input([conf.CONTEXT_WIN]))
+        model_list.insert(
+            1, tf.keras.layers.Lambda(lambda t: tf.expand_dims(t, -1))
+        )
+        model_list.insert(2, front_end.MelSpectrogram())
+        self.model = tf.keras.Sequential(
+            layers=[layer for layer in model_list]
+        )
+
+
+class HumpBackNorthAtlantic(ModelHelper):
+    """
+    Defualt class for North Atlantic Humpback Whale Song detection. If no new
+    training is performed this class is the default class. The model is
+    currently included in the repository. The model will be extracted
+    and made ready for use.
+
+    Parameters
+    ----------
+    ModelHelper : class
+        helper class providing necessary functionalities
+    """
+
+    def __init__(self, **kwargs):
+        pass
+
+    def load_model(self, **kwargs):
+        if not Path(conf.MODEL_DIR).joinpath(conf.MODEL_NAME).exists():
+            for model_path in Path(conf.MODEL_DIR).iterdir():
+                if not model_path.suffix == ".zip":
+                    continue
+                else:
+                    with zipfile.ZipFile(model_path, "r") as model_zip:
+                        model_zip.extractall(conf.MODEL_DIR)
+        self.model = tf.keras.models.load_model(
+            Path(conf.MODEL_DIR).joinpath(conf.MODEL_NAME),
+            custom_objects={"FBetaScote": metrics.FBetaScore},
+        )
+
+
+class GoogleMod(ModelHelper):  # TODO change name
+    def __init__(self, **params) -> None:
+        """
+        This class is the framework to load and flatten the model created
+        by Matthew Harvey in collaboration with Ann Allen for the
+        PIFSC HARP data
+        (https://www.frontiersin.org/article/10.3389/fmars.2021.607321).
+
+        Args:
+            params (dict): model parameters
+        """
+        self.load_google_new(**params)
+        self.load_flat_model(**params)
+
+    def load_google_new(self, load_g_ckpt=conf.LOAD_G_CKPT, **_):
+        """
+        Load google model architecture. By default the model weights are
+        initiated with the pretrained weights from the google checkpoint.
+
+        Args:
+            load_g_ckpt (bool, optional): Initialize model weights with Google
+            pretrained weights. Defaults to True.
+        """
+        self.model = humpback_model.Model()
+        if load_g_ckpt:
+            self.model = self.model.load_from_tf_hub()
+
+    def load_flat_model(self, input_tensors="spectrograms", **_):
+        """
+        Take nested model architecture from Harvey Matthew and flatten it for
+        ease of use. This way trainability of layers can be iteratively
+        defined. The model still has a nested structure. The ResNet blocks are
+        combined into layers of type Block, but because their trainability would
+        only be changed on the block level, this degree of nesting shouldn't
+        complicate the usage of the model.
+        By default the model is itiated with spectrograms of shape [128, 64] as
+        inputs. This means that spectrograms have to be precomputed.
+        Alternatively if the argument input_tensors is set to something else,
+        inputs are assumed to be audio arrays of 39124 samples length.
+        As this process is very specific to the model ascertained from
+        Mr. Harvey, layer indices are hard coded.
+
+        Args:
+            input_tensors (str): input type, if not spectrograms, they are
+            assumed to be audio arrays of 39124 samples length.
+            Defaults to 'spectrograms'.
+        """
+        # create list which will contain all the layers
+        model_list = []
+        if input_tensors == "spectrograms":
+            # add Input layer
+            model_list.append(tf.keras.layers.Input([128, 64]))
+        else:
+            # add MelSpectrogram layer
+            model_list.append(tf.keras.layers.Input([conf.CONTEXT_WIN]))
+            model_list.append(
+                tf.keras.layers.Lambda(lambda t: tf.expand_dims(t, -1))
+            )
+            model_list.append(self.model.layers[0])
+
+        # add PCEN layer
+        model_list.append(self.model.layers[1])
+        # iterate through PreBlocks
+        model_list.append(self.model.layers[2]._layers[0])
+        for layer in self.model.layers[2]._layers[1]._layers:
+            model_list.append(layer)
+        # change name, to make sure every layer has a unique name
+        num_preproc_layers = len(model_list)
+        model_list[num_preproc_layers - 1]._name = "pool_pre_resnet"
+        # iterate over ResNet blocks
+        c = 0
+        for i, high_layer in enumerate(self.model.layers[2]._layers[2:6]):
+            for j, layer in enumerate(high_layer._layers):
+                c += 1
+                model_list.append(layer)
+                model_list[num_preproc_layers - 1 + c]._name += f"_{i}"
+        # add final Dense layers
+        model_list.append(self.model.layers[2]._layers[-1])
+        model_list.append(self.model.layers[-1])
+        # normalize results between 0 and 1
+        model_list.append(tf.keras.layers.Activation("sigmoid"))
+
+        # generate new model
+        self.model = tf.keras.Sequential(
+            layers=[layer for layer in model_list]
+        )
+
+
+class KerasAppModel(ModelHelper):
+    """
+    Class providing functionalities for usage of standard keras application
+    models like EfficientNet. The keras application name is passed and
+    the helper class is then used in case existing checkpoints need to be
+    loaded or the shape of the input array needs change.
+
+    Parameters
+    ----------
+    ModelHelper : class
+        helper class providing necessary functionalities
+    """
+
+    def __init__(self, keras_mod_name="EfficientNetB0", **params) -> None:
+        keras_model = getattr(tf.keras.applications, keras_mod_name)(
+            include_top=True,
+            weights=None,
+            input_tensor=None,
+            input_shape=[128, 64, 3],
+            pooling=None,
+            classes=1,
+            classifier_activation="sigmoid",
+        )
+
+        self.model = tf.keras.Sequential(
+            [
+                tf.keras.layers.Input([128, 64]),
+                leaf_pcen.PCEN(
+                    alpha=0.98,
+                    delta=2.0,
+                    root=2.0,
+                    smooth_coef=0.025,
+                    floor=1e-6,
+                    trainable=True,
+                    name="pcen",
+                ),
+                # tf.keras.layers.Lambda(lambda t: 255. *t /tf.math.reduce_max(t)),
+                tf.keras.layers.Lambda(
+                    lambda t: tf.tile(
+                        tf.expand_dims(t, -1), [1 for _ in range(3)] + [3]
+                    )
+                ),
+                keras_model,
+            ]
+        )
+
+
+def init_model(
+    model_name: str = conf.MODELCLASSNAME,
+    training_path: str = conf.LOAD_CKPT_PATH,
+    input_specs=False,
+    **kwargs,
+) -> tf.keras.Sequential:
+    """
+    Initiate model instance, load weights. As the model is trained on
+    spectrogram tensors but will now be used for inference on audio files
+    containing continuous audio arrays, the input shape of the model is
+    changed after the model weights have been loaded.
+
+    Parameters
+    ----------
+    model_instance : type
+        callable class to create model object
+    training_path : str
+        checkpoint path
+
+    Returns
+    -------
+    tf.keras.Sequential
+        the sequential model with pretrained weights
+    """
+    model_class = getattr(sys.modules[__name__], model_name)
+    mod_obj = model_class(**kwargs)
+    if conf.MODEL_NAME == "FlatHBNA":
+        input_specs = True
+    if model_class == HumpBackNorthAtlantic:
+        mod_obj.load_model()
+    else:
+        mod_obj.load_ckpt(training_path)
+    if not input_specs:
+        mod_obj.change_input_to_array()
+    return mod_obj.model
+
+
+def get_labels_and_preds(
+    model_name: str, training_path: str, val_data: tf.data.Dataset, **kwArgs
+) -> tuple:
+    """
+    Retrieve labels and predictions of validation set and given model
+    checkpoint.
+
+    Parameters
+    ----------
+    model_instance : type
+        model class
+    training_path : str
+        path to checkpoint
+    val_data : tf.data.Dataset
+        validation dataset
+
+    Returns
+    -------
+    labels: np.ndarray
+        labels
+    preds: mp.ndarray
+        predictions
+    """
+    model = init_model(
+        load_from_ckpt=True,
+        model_name=model_name,
+        training_path=training_path,
+        **kwArgs,
+    )
+    preds = model.predict(x=prep_ds_4_preds(val_data))
+    labels = get_val_labels(val_data, len(preds))
+    return labels, preds
+
+
+def prep_ds_4_preds(dataset):
+    """
+    Prepare dataset for prediction, by batching and ensuring that only
+    arrays and corresponding labels are passed (necessary because if
+    data about the origin of the array is passed, i.e. the file path and
+    start time of the array within that file, model.predict fails.).
+
+    Parameters
+    ----------
+    dataset : TfRecordDataset
+        dataset
+
+    Returns
+    -------
+    TFRecordDataset
+        prepared dataset
+    """
+    if len(list(dataset.take(1))[0]) > 2:
+        return dataset.map(lambda x, y, z, w: (x, y)).batch(batch_size=32)
+    else:
+        return dataset.batch(batch_size=32)

acodet/plot_utils.py

@@ -0,0 +1,466 @@
+import numpy as np
+import matplotlib.pyplot as plt
+import matplotlib.colors as mcolors
+from matplotlib.gridspec import GridSpec
+import time
+from pathlib import Path
+import seaborn as sns
+import json
+import librosa as lb
+from . import global_config as conf
+from . import funcs
+from . import tfrec
+from . import models
+import tensorflow as tf
+
+drop_keyz = {"fbeta", "val_fbeta"}
+sns.set_theme()
+sns.set_style("white")
+
+
+def plot_model_results(
+    datetimes, labels=None, fig=None, legend=True, **kwargs
+):
+    plt.rc("axes", labelsize=20)
+    plt.rc("axes", titlesize=20)
+    plt.rc("xtick", labelsize=14)
+    plt.rc("ytick", labelsize=14)
+    if fig == None:
+        fig = plt.figure(figsize=(15, 8))
+        savefig = True
+    else:
+        savefig = False
+
+    if not isinstance(datetimes, list):
+        datetimes = [datetimes]
+
+    checkpoint_paths = []
+    for datetime in datetimes:
+        checkpoint_paths += list(
+            Path(f"../trainings/{datetime}").glob("unfreeze_*")
+        )
+
+    r, c = 2, 5
+    for j, checkpoint_path in enumerate(checkpoint_paths):
+        unfreeze = checkpoint_path.stem.split("_")[-1]
+
+        if not Path(f"{checkpoint_path}/results.json").exists():
+            if j == 0:
+                ax = fig.subplots(ncols=c, nrows=r)
+            continue
+        with open(f"{checkpoint_path}/results.json", "r") as f:
+            results = json.load(f)
+            for k in drop_keyz:
+                del results[k]
+
+        if j == 0:
+            c = len(list(results.keys())) // 2
+            ax = fig.subplots(ncols=c, nrows=r)
+
+        if not labels is None:
+            label = labels[j]
+        else:
+            label = f"{checkpoint_path.parent.stem}_{unfreeze}"
+
+        for i, key in enumerate(results.keys()):
+            if "val_" in key:
+                row = 1
+            else:
+                row = 0
+            if "loss" in key or i == 0:
+                col = 0
+            else:
+                col += 1
+
+            ax[row, col].plot(results[key], label=label)
+            if not col == 0:
+                ax[row, col].set_ylim([0.7, 1.01])
+
+            # axis handling depending on subplot index
+            if row == 1 and col == 0:
+                ax[row, col].set_ylim([0, 1.01])
+            if row == 0 and j == 0:
+                ax[row, col].set_title(f"{key}")
+            if row == col == 0:
+                ax[row, col].set_ylim([0, 0.5])
+                ax[row, col].set_ylabel("training")
+            elif row == 1 and col == 0:
+                ax[row, col].set_ylabel("val")
+            if legend and row == 1 and col == c - 1:
+                ax[row, col].legend(loc="center left", bbox_to_anchor=(1, 0.5))
+
+    info_string = ""
+    for key, val in kwargs.items():
+        info_string += f" | {key}: {val}"
+
+    ref_time = time.strftime("%Y%m%d", time.gmtime())
+    if savefig:
+        fig.tight_layout()
+        fig.savefig(f"../trainings/{datetime}/model_results_{ref_time}.png")
+    else:
+        return fig
+
+
+def plot_spec_from_file(file, start, sr, cntxt_wn_sz=39124, **kwArgs):
+    audio, sr = lb.load(
+        file, sr=sr, offset=start / sr, duration=cntxt_wn_sz / sr
+    )
+    return simple_spec(audio, sr=sr, cntxt_wn_sz=cntxt_wn_sz, **kwArgs)
+
+
+def plot_sample_spectrograms(
+    dataset,
+    *,
+    dir,
+    name,
+    ds_size=None,
+    random=True,
+    seed=None,
+    sr=conf.SR,
+    rows=4,
+    cols=4,
+    plot_meta=False,
+    **kwargs,
+):
+    r, c = rows, cols
+    if isinstance(dataset, tf.data.Dataset):
+        if random:
+            if ds_size is None:
+                ds_size = sum(1 for _ in dataset)
+            np.random.seed(seed)
+            rand_skip = np.random.randint(ds_size)
+            sample = dataset.skip(rand_skip).take(r * c)
+        else:
+            sample = dataset.take(r * c)
+    elif isinstance(dataset, list):
+        sample = dataset
+
+    max_freq_bin = 128 // (conf.SR // 2000)
+
+    fmin = sr / 2 / next(iter(sample))[0].numpy().shape[0]
+    fmax = sr / 2 / next(iter(sample))[0].numpy().shape[0] * max_freq_bin
+    fig, axes = plt.subplots(nrows=r, ncols=c, figsize=[12, 10])
+
+    for i, (aud, *lab) in enumerate(sample):
+        if i == r * c:
+            break
+        ar = aud.numpy()[:, 1:max_freq_bin].T
+        axes[i // r][i % c].imshow(
+            ar, origin="lower", interpolation="nearest", aspect="auto"
+        )
+        if len(lab) == 1:
+            axes[i // r][i % c].set_title(f"label: {lab[0]}")
+        elif len(lab) == 3:
+            label, file, t = (v.numpy() for v in lab)
+            axes[i // r][i % c].set_title(
+                f"label: {label}; t in f: {funcs.get_time(t)}\n"
+                f"file: {Path(file.decode()).stem}"
+            )
+        if i // r == r - 1 and i % c == 0:
+            axes[i // r][i % c].set_xticks(np.linspace(0, ar.shape[1], 5))
+            xlabs = np.linspace(0, 3.9, 5).astype(str)
+            axes[i // r][i % c].set_xticklabels(xlabs)
+            axes[i // r][i % c].set_xlabel("time in s")
+            axes[i // r][i % c].set_yticks(np.linspace(0, ar.shape[0] - 1, 7))
+            ylabs = np.linspace(fmin, fmax, 7).astype(int).astype(str)
+            axes[i // r][i % c].set_yticklabels(ylabs)
+            axes[i // r][i % c].set_ylabel("freq in Hz")
+        else:
+            axes[i // r][i % c].set_xticks([])
+            axes[i // r][i % c].set_xticklabels([])
+            axes[i // r][i % c].set_yticks([])
+            axes[i // r][i % c].set_yticklabels([])
+
+    fig.suptitle(f"{name} sample of 16 spectrograms. random={random}")
+    fig.savefig(f"../trainings/{dir}/{name}_sample.png")
+
+
+def simple_spec(
+    signal,
+    ax=None,
+    fft_window_length=2**11,
+    sr=10000,
+    cntxt_wn_sz=39124,
+    fig=None,
+    colorbar=True,
+):
+    S = np.abs(lb.stft(signal, win_length=fft_window_length))
+    if not ax:
+        fig_new, ax = plt.subplots()
+    if fig:
+        fig_new = fig
+    # limit S first dimension from [10:256], thatway isolating frequencies
+    # (sr/2)/1025*10 = 48.78 to (sr/2)/1025*266 = 1297.56 for visualization
+    fmin = sr / 2 / S.shape[0] * 10
+    fmax = sr / 2 / S.shape[0] * 266
+    S_dB = lb.amplitude_to_db(S[10:266, :], ref=np.max)
+    img = lb.display.specshow(
+        S_dB,
+        x_axis="s",
+        y_axis="linear",
+        sr=sr,
+        win_length=fft_window_length,
+        ax=ax,
+        x_coords=np.linspace(0, cntxt_wn_sz / sr, S_dB.shape[1]),
+        y_coords=np.linspace(fmin, fmax, 2**8),
+        vmin=-60,
+    )
+
+    if colorbar:
+        fig_new.colorbar(img, ax=ax, format="%+2.0f dB")
+        return fig_new, ax
+    else:
+        return ax
+
+
+def plot_conf_matr(labels, preds, ax, iteration, title, **kwargs):
+    plt.rc("axes", titlesize=40)
+    plt.rc("font", size=32)
+    bin_preds = list(map(lambda x: 1 if x >= conf.THRESH else 0, preds))
+    heat = tf.math.confusion_matrix(labels, bin_preds).numpy()
+    rearrange = lambda x: np.array([[x[1, 1], x[1, 0]], [x[0, 1], x[0, 0]]])
+    # rearranged_head = rearrange(heat)
+    value_string = "{}\n{:.0f}%"
+    heat_annot = [[[], []], [[], []]]
+    heat_perc = [[[], []], [[], []]]
+    for row in range(2):
+        for col in range(2):
+            heat_annot[row][col] = value_string.format(
+                heat[row, col], heat[row, col] / np.sum(heat[row]) * 100
+            )
+            heat_perc[row][col] = heat[row, col] / np.sum(heat[row]) * 100
+    rearranged_annot = rearrange(np.array(heat_annot))
+    rearranged_heat = rearrange(np.array(heat_perc))
+
+    ax = sns.heatmap(
+        rearranged_heat,
+        annot=rearranged_annot,
+        fmt="",
+        cbar=False,
+        ax=ax,
+        xticklabels=False,
+        yticklabels=False,
+    )
+    ax.set_yticks([0.5, 1.5], labels=["TP", "TN"], fontsize=32)
+    ax.set_xticks([1.5, 0.5], labels=["pred. N", "pred.P"], fontsize=32)
+    color = list(sns.color_palette())[iteration]
+    ax.set_title(title, color=color)
+    return ax
+
+
+def plot_pr_curve(
+    labels, preds, ax, training_path, iteration=0, legend=True, **kwargs
+):
+    m = dict()
+    for met in ("Recall", "Precision"):
+        threshs = list(np.linspace(0, 1, num=200)[:-1])
+        m.update(
+            {met: funcs.get_pr_arrays(labels, preds, met, thresholds=threshs)}
+        )
+    for curve in ("ROC", "PR"):
+        m.update(
+            {curve: funcs.get_pr_arrays(labels, preds, "AUC", curve=curve)}
+        )
+    perform_str = f"; AUC_PR:{m['PR']:.2f}; AUC_ROC:{m['ROC']:.2f}"
+    print(
+        "p_.5: ",
+        m["Precision"][100],
+        "\nr_.5: ",
+        m["Recall"][100],
+        "\nAUC-PR: ",
+        m["PR"],
+        "\nAUC-ROC: ",
+        m["ROC"],
+    )
+    for k, i in m.items():
+        m[k] = i.astype(float)
+    m["Recall"] = list(m["Recall"])
+    m["Precision"] = list(m["Precision"])
+    with open(f"../perform_metrics_{training_path.stem}.json", "w") as f:
+        json.dump(m, f)
+    print(perform_str)
+    if "plot_labels" in kwargs:
+        if isinstance(kwargs["plot_labels"], list):
+            label = kwargs["plot_labels"][iteration] + perform_str
+        else:
+            label = kwargs["plot_labels"] + perform_str
+    elif "load_untrained_model" in kwargs:
+        label = "untrained_model" + perform_str
+    else:
+        label = str(
+            training_path.parent.stem
+            + training_path.stem.split("_")[-1]
+            + perform_str
+        )
+
+    ax.plot(m["Recall"], m["Precision"], label=label)
+
+    ax.set_ylabel("precision", fontsize=32)
+    ax.set_xlabel("recall", fontsize=32)
+    ax.set_ylim([0, 1])
+    ax.set_xlim([0, 1])
+    ax.set_xticks([0, 0.3, 0.7, 1])
+    ax.set_yticks([0, 0.3, 0.7, 1])
+    ax.set_xticklabels(["0", "0.3", "0.7", "1"], fontsize=24)
+    ax.set_yticklabels(["0", "0.3", "0.7", "1"], fontsize=24)
+    if legend:
+        ax.legend()
+    ax.grid(True)
+    ax.set_title("PR Curve", fontsize=32)
+    return ax
+
+
+def plot_evaluation_metric(
+    model_name,
+    training_runs,
+    val_data,
+    fig,
+    plot_pr=True,
+    plot_cm=False,
+    plot_untrained=False,
+    titles=None,
+    keras_mod_name=False,
+    **kwargs,
+):
+    r = plot_cm + plot_pr
+    c = len(training_runs)
+    if c < 1:
+        c = 1
+    gs = GridSpec(r, c, figure=fig)
+    if plot_pr:
+        ax_pr = fig.add_subplot(gs[0, :])
+
+    for i, run in enumerate(training_runs):
+        if not isinstance(model_name, list):
+            model_name = [model_name]
+        if not isinstance(keras_mod_name, list):
+            keras_mod_name = [keras_mod_name]
+        if not isinstance(titles, list):
+            title = Path(run).parent.stem + Path(run).stem.split("_")[-1]
+        else:
+            title = titles[i]
+        labels, preds = models.get_labels_and_preds(
+            model_name[i],
+            run,
+            val_data,
+            keras_mod_name=keras_mod_name[i],
+            **kwargs,
+        )
+        if not plot_pr:
+            plot_conf_matr(
+                labels,
+                preds,
+                fig.add_subplot(gs[-1, i]),
+                title=title,
+                iteration=i,
+            )
+        else:
+            ax_pr = plot_pr_curve(
+                labels, preds, ax_pr, run, iteration=i, **kwargs
+            )
+            if plot_untrained:
+                ax_pr = plot_pr_curve(
+                    labels,
+                    preds,
+                    ax_pr,
+                    run,
+                    load_untrained_model=True,
+                    iteration=i,
+                    **kwargs,
+                )
+            if plot_cm:
+                plot_conf_matr(
+                    labels,
+                    preds,
+                    fig.add_subplot(gs[-1, i]),
+                    title=title,
+                    iteration=i,
+                )
+
+        print("creating pr curve for ", run.stem)
+
+    if "legend" in kwargs and kwargs["legend"]:
+        ax_pr.legend(loc="center left", bbox_to_anchor=(1, 0.5))
+    return fig
+
+
+def create_and_save_figure(
+    model_name,
+    tfrec_path,
+    batch_size,
+    train_date,
+    debug=False,
+    plot_pr=True,
+    plot_cm=False,
+    **kwargs,
+):
+    training_runs = list(Path(f"../trainings/{train_date}").glob("unfreeze*"))
+    val_data = tfrec.run_data_pipeline(tfrec_path, "val", return_spec=False)
+
+    fig = plt.figure(constrained_layout=True)
+
+    plot_evaluation_metric(
+        model_name,
+        training_runs,
+        val_data,
+        fig=fig,
+        plot_pr=plot_pr,
+        plot_cm=plot_cm,
+        **kwargs,
+    )
+
+    info_string = ""
+    for key, val in kwargs.items():
+        info_string += f" | {key}: {val}"
+    fig.suptitle(f"Evaluation Metrics{info_string}")
+
+    fig.savefig(f"{training_runs[0].parent}/eval_metrics.png")
+
+
+def plot_pre_training_spectrograms(
+    train_data, test_data, augmented_data, time_start, seed
+):
+    plot_sample_spectrograms(
+        train_data, dir=time_start, name="train_all", seed=seed
+    )
+    for i, (augmentation, aug_name) in enumerate(augmented_data):
+        plot_sample_spectrograms(
+            augmentation,
+            dir=time_start,
+            name=f"augment_{i}-{aug_name}",
+            seed=seed,
+        )
+    plot_sample_spectrograms(test_data, dir=time_start, name="test")
+
+
+def compare_random_spectrogram(filenames, dataset_size=conf.TFRECS_LIM):
+    r = np.random.randint(dataset_size)
+    dataset = (
+        tf.data.TFRecordDataset(filenames)
+        .map(tfrec.parse_tfrecord_fn)
+        .skip(r)
+        .take(1)
+    )
+
+    sample = next(iter(dataset))
+    aud, file, lab, time = (sample[k].numpy() for k in list(sample.keys()))
+    file = file.decode()
+
+    fig, ax = plt.subplots(ncols=2, figsize=[12, 8])
+    ax[0] = funcs.simple_spec(
+        aud, fft_window_length=512, sr=10000, ax=ax[0], colorbar=False
+    )
+    _, ax[1] = funcs.plot_spec_from_file(
+        file, ax=ax[1], start=time, fft_window_length=512, sr=10000, fig=fig
+    )
+    ax[0].set_title(
+        f"Spec of audio sample from \ntfrecords array nr. {r}"
+        f" | label: {lab}"
+    )
+    ax[1].set_title(
+        f"Spec of audio sample from file: \n{Path(file).stem}"
+        f" | time in file: {funcs.get_time(time/10000)}"
+    )
+
+    fig.suptitle("Comparison between tfrecord audio and file audio")
+    fig.savefig(f"{tfrec.TFRECORDS_DIR}_check_imgs/comp_{Path(file).stem}.png")

acodet/split_daily_annots.py

@@ -0,0 +1,39 @@
+import numpy as np
+import pandas as pd
+from pathlib import Path
+from hbdet.funcs import get_dt_filename
+
+
+def create_day_dir(file_path):
+    file_path.parent.joinpath(f.stem).mkdir(exist_ok=True)
+
+
+main_path = Path("../Annais/Blue_whales/Annotations_bluewhales")
+
+files = list(main_path.rglob("2*.txt"))
+counter = 0
+for f in files:
+    data = pd.read_csv(f, sep="\t")
+    data = data[data["Comments"] == "S"]
+    if not "Begin File" in data.columns:
+        continue
+    else:
+        create_day_dir(f)
+        for beg_f in np.unique(data["Begin File"]):
+            hour_data = data[data["Begin File"] == beg_f]
+            hour = get_dt_filename(beg_f).hour
+            new_data = hour_data.copy()
+            new_data["Begin Time (s)"] -= hour * 1500
+            new_data["End Time (s)"] -= hour * 1500
+            save_str = f.parent.joinpath(f.stem).joinpath(
+                Path(beg_f).stem + "_annotated.txt"
+            )
+            new_data.to_csv(save_str, sep="\t")
+            counter += 1
+            print(
+                "most recent file:",
+                f.stem,
+                "files written: ",
+                counter,
+                end="\r",
+            )

acodet/tfrec.py

@@ -0,0 +1,434 @@
+from tokenize import Intnumber
+from functools import partial
+import numpy as np
+import pandas as pd
+import tensorflow as tf
+from . import funcs
+import random
+from .humpback_model_dir import front_end
+from pathlib import Path
+import json
+from . import global_config as conf
+
+########################################################
+#################  WRITING   ###########################
+########################################################
+
+
+def exclude_files_from_dataset(annots):
+    """
+    Because some of the calls are very faint, a number of files are rexcluded
+    from the dataset to make sure that the model performance isn't obscured
+    by poor data.
+
+    Args:
+        annots (pd.DataFrame): annotations
+
+    Returns:
+        pd.DataFrame: cleaned annotation dataframe
+    """
+    exclude_files = [
+        "180324160003",
+        "PAM_20180323",
+        "PAM_20180324",
+        "PAM_20180325_0",
+        "PAM_20180325_17",
+        "PAM_20180326",
+        "PAM_20180327",
+        "PAM_20180329",
+        "PAM_20180318",
+        "PAM_20190321",
+        "20022315",
+        "20022621",
+        "210318",
+        "210319",
+        "210327",
+    ]
+    drop_files = []
+    for file in np.unique(annots.filename):
+        for exclude in exclude_files:
+            if exclude in file:
+                drop_files.append(file)
+    annots.index = annots.filename
+
+    return annots.drop(annots.loc[drop_files].index), annots.loc[drop_files]
+
+
+def audio_feature(list_of_floats):
+    """
+    Returns a list of floats.
+
+    Args:
+        list_of_floats (list): list of floats
+
+    Returns:
+        tf.train.Feature: tensorflow feature containing a list of floats
+    """
+    return tf.train.Feature(
+        float_list=tf.train.FloatList(value=list_of_floats)
+    )
+
+
+def int_feature(value):
+    """
+    Returns a int value.
+
+    Args:
+        value (int): label value
+
+    Returns:
+        tf.train.Feature: tensorflow feature containing a int
+    """
+    return tf.train.Feature(int64_list=tf.train.Int64List(value=[int(value)]))
+
+
+def string_feature(value):
+    """
+    Returns a bytes array.
+
+    Args:
+        value (string): path to file
+
+    Returns:
+        tf.train.Feature: tensorflow feature containing a bytes array
+    """
+    return tf.train.Feature(
+        bytes_list=tf.train.BytesList(value=[bytes(value, "utf-8")])
+    )
+
+
+def create_example(audio, label, file, time):
+    """
+    Create a tensorflow Example object containing the tf Features that will
+    be saved in one file. The file will contain the audio data, corresponding
+    label, the file path corresponding to the audio, and the time in samples
+    where in the file that audio data begins. The audio data is saved
+    according to the frame rate in params, for the Google model 10 kHz.
+    The label is binary, either 0 for noise or 1 for call.
+
+    Args:
+        audio (list): raw audio data of length params['cntxt_wn_sz']
+        label (int): either 1 for call or 0 for noise
+        file (string): path of file corresponding to audio data
+        time (int): time in samples when audio data begins within file
+
+    Returns:
+        tf.train.Example: Example object containing all data
+    """
+    feature = {
+        "audio": audio_feature(audio),
+        "label": int_feature(label),
+        "file": string_feature(file),
+        "time": int_feature(time),
+    }
+    return tf.train.Example(features=tf.train.Features(feature=feature))
+
+
+def read_raw_file(file, annots, **kwargs):
+    """
+    Load annotations for file, correct annotation starting times to make sure
+    that the signal is in the window center.
+
+    Args:
+        file (string): path to file
+
+    Returns:
+        tuple: audio segment arrays, label arrays and time arrays
+    """
+
+    file_annots = funcs.get_annots_for_file(annots, file)
+
+    x_call, x_noise, times_c, times_n = funcs.cntxt_wndw_arr(
+        file_annots, file, **kwargs
+    )
+    y_call = np.ones(len(x_call), dtype="float32")
+    y_noise = np.zeros(len(x_noise), dtype="float32")
+
+    return (x_call, y_call, times_c), (x_noise, y_noise, times_n)
+
+
+def write_tfrecords(annots, save_dir, inbetween_noise=True, **kwargs):
+    """
+    Write tfrecords files from wav files.
+    First the files are imported and the noise files are generated. After that
+    a loop iterates through the tuples containing the audio arrays, labels, and
+    startint times of the audio arrays within the given file. tfrecord files
+    contain no more than 600 audio arrays. The respective audio segments,
+    labels, starting times, and file paths are saved in the files.
+
+    Args:
+        files (list): list of file paths to the audio files
+    """
+    files = np.unique(annots.filename)
+
+    if len(files) == 0:
+        return
+
+    random.shuffle(files)
+
+    split_mode = "within_file"
+    specs = ["size", "noise", "calls"]
+    folders = ["train", "test", "val"]
+    train_file_index = int(len(files) * conf.TRAIN_RATIO)
+    test_file_index = int(
+        len(files) * (1 - conf.TRAIN_RATIO) * conf.TEST_VAL_RATIO
+    )
+
+    dataset = {k: {k1: 0 for k1 in folders} for k in specs}
+    data_meta_dict = dict({"data_split": split_mode})
+    files_dict = {}
+    tfrec_num = 0
+    for i, file in enumerate(files):
+        print(
+            "writing tf records files, progress:" f"{i/len(files)*100:.0f} %"
+        )
+
+        if i < train_file_index:
+            folder = folders[0]
+        elif i < train_file_index + test_file_index:
+            folder = folders[1]
+        else:
+            folder = folders[2]
+
+        call_tup, noise_tup = read_raw_file(
+            file, annots, inbetween_noise=inbetween_noise, **kwargs
+        )
+
+        calls = randomize_arrays(call_tup, file)
+        noise = randomize_arrays(noise_tup, file)
+        samples = [*calls, *noise]
+        random.shuffle(samples)
+        data = dict()
+
+        end_tr, end_te = map(
+            lambda x: int(x * len(samples)),
+            (
+                conf.TRAIN_RATIO,
+                (1 - conf.TRAIN_RATIO) * conf.TEST_VAL_RATIO
+                + conf.TRAIN_RATIO,
+            ),
+        )
+
+        data["train"] = samples[:end_tr]
+        data["test"] = samples[end_tr:end_te]
+        data["val"] = samples[end_tr:-1]
+
+        for folder, samples in data.items():
+            split_by_max_length = [
+                samples[j * conf.TFRECS_LIM : (j + 1) * conf.TFRECS_LIM]
+                for j in range(len(samples) // conf.TFRECS_LIM + 1)
+            ]
+            for samps in split_by_max_length:
+                tfrec_num += 1
+                writer = get_tfrecords_writer(
+                    tfrec_num, folder, save_dir, **kwargs
+                )
+                files_dict, dataset = update_dict(
+                    samps, files_dict, dataset, folder, tfrec_num
+                )
+
+                for audio, label, file, time in samps:
+                    examples = create_example(audio, label, file, time)
+                    writer.write(examples.SerializeToString())
+
+    # TODO automatisch die noise sachen miterstellen
+    data_meta_dict.update({"dataset": dataset})
+    data_meta_dict.update({"files": files_dict})
+    path = add_child_dirs(save_dir, **kwargs)
+    with open(path.joinpath(f"dataset_meta_{folders[0]}.json"), "w") as f:
+        json.dump(data_meta_dict, f)
+
+
+def randomize_arrays(tup, file):
+    x, y, times = tup
+
+    rand = np.arange(len(x))
+    random.shuffle(rand)
+
+    return zip(x[rand], y[rand], [file] * len(x), np.array(times)[rand])
+
+
+def update_dict(samples, d, dataset_dict, folder, tfrec_num):
+    calls = sum(1 for i in samples if i[1] == 1)
+    noise = sum(1 for i in samples if i[1] == 0)
+    size = noise + calls
+    d.update(
+        {f"file_%.2i_{folder}" % tfrec_num: k for k in [size, noise, calls]}
+    )
+    for l, k in zip(("size", "calls", "noise"), (size, calls, noise)):
+        dataset_dict[l][folder] += k
+    return d, dataset_dict
+
+
+def add_child_dirs(save_dir, alt_subdir="", all_noise=False, **kwargs):
+    path = save_dir.joinpath(alt_subdir)
+    if all_noise:
+        path = path.joinpath("noise")
+    return path
+
+
+def get_tfrecords_writer(num, fold, save_dir, **kwargs):
+    """
+    Return TFRecordWriter object to write file.
+
+    Args:
+        num (int): file number
+
+    Returns:
+        TFRecordWriter object: file handle
+    """
+    path = add_child_dirs(save_dir, **kwargs)
+    Path(path.joinpath(fold)).mkdir(parents=True, exist_ok=True)
+    return tf.io.TFRecordWriter(
+        str(path.joinpath(fold).joinpath("file_%.2i.tfrec" % num))
+    )
+
+
+def get_src_dir_structure(file, annot_dir):
+    """
+    Return the directory structure of a file relative to the annotation
+    directory. This enables the caller to build a directory structure
+    identical to the directory structure of the source files.
+
+    Parameters
+    ----------
+    file : pathlib.Path
+        file path
+    annot_dir : str
+        path to annotation directory
+
+    Returns
+    -------
+    pathlike
+        directory structure
+    """
+    if len(list(file.relative_to(annot_dir).parents)) == 1:
+        return file.relative_to(annot_dir).parent
+    else:
+        return list(file.relative_to(annot_dir).parents)[-2]
+
+
+def write_tfrec_dataset(annot_dir=conf.ANNOT_DEST, active_learning=True):
+    annotation_files = list(Path(annot_dir).glob("**/*.csv"))
+    if active_learning:
+        inbetween_noise = False
+    else:
+        inbetween_noise = True
+
+    for file in annotation_files:
+        annots = pd.read_csv(file)
+        if "explicit_noise" in file.stem:
+            all_noise = True
+        else:
+            all_noise = False
+
+        save_dir = Path(conf.TFREC_DESTINATION).joinpath(
+            get_src_dir_structure(file, annot_dir)
+        )
+
+        write_tfrecords(
+            annots,
+            save_dir,
+            all_noise=all_noise,
+            inbetween_noise=inbetween_noise,
+        )
+
+
+########################################################
+#################  READING   ###########################
+########################################################
+
+
+def parse_tfrecord_fn(example):
+    """
+    Parser for tfrecords files.
+
+    Args:
+        example (tf.train.Example instance): file containing 4 features
+
+    Returns:
+        tf.io object: tensorflow object containg the data
+    """
+    feature_description = {
+        "audio": tf.io.FixedLenFeature([conf.CONTEXT_WIN], tf.float32),
+        "label": tf.io.FixedLenFeature([], tf.int64),
+        "file": tf.io.FixedLenFeature([], tf.string),
+        "time": tf.io.FixedLenFeature([], tf.int64),
+    }
+    return tf.io.parse_single_example(example, feature_description)
+
+
+def prepare_sample(features, return_meta=False, **kwargs):
+    if not return_meta:
+        return features["audio"], features["label"]
+    else:
+        return (
+            features["audio"],
+            features["label"],
+            features["file"],
+            features["time"],
+        )
+
+
+def get_dataset(filenames, AUTOTUNE=None, **kwargs):
+    dataset = (
+        tf.data.TFRecordDataset(filenames, num_parallel_reads=AUTOTUNE)
+        .map(parse_tfrecord_fn, num_parallel_calls=AUTOTUNE)
+        .map(partial(prepare_sample, **kwargs), num_parallel_calls=AUTOTUNE)
+    )
+    return dataset
+
+
+def run_data_pipeline(
+    root_dir, data_dir, AUTOTUNE=None, return_spec=True, **kwargs
+):
+    if not isinstance(root_dir, list):
+        root_dir = [root_dir]
+    files = []
+    for root in root_dir:
+        if data_dir == "train":
+            files += tf.io.gfile.glob(f"{root}/{data_dir}/*.tfrec")
+        elif data_dir == "noise":
+            files += tf.io.gfile.glob(f"{root}/{data_dir}/*.tfrec")
+            files += tf.io.gfile.glob(f"{root}/noise/train/*.tfrec")
+        else:
+            try:
+                files += tf.io.gfile.glob(f"{root}/noise/{data_dir}/*.tfrec")
+            except:
+                print("No explicit noise for ", root)
+            files += tf.io.gfile.glob(f"{root}/{data_dir}/*.tfrec")
+    dataset = get_dataset(files, AUTOTUNE=AUTOTUNE, **kwargs)
+    if return_spec:
+        return make_spec_tensor(dataset)
+    else:
+        return dataset
+
+
+def spec():
+    return tf.keras.Sequential(
+        [
+            tf.keras.layers.Input([conf.CONTEXT_WIN]),
+            tf.keras.layers.Lambda(lambda t: tf.expand_dims(t, -1)),
+            front_end.MelSpectrogram(),
+        ]
+    )
+
+
+def prepare(
+    ds,
+    batch_size,
+    shuffle=False,
+    shuffle_buffer=750,
+    augmented_data=None,
+    AUTOTUNE=None,
+):
+    if shuffle:
+        ds = ds.shuffle(shuffle_buffer)
+    ds = ds.batch(batch_size)
+    return ds.prefetch(buffer_size=AUTOTUNE)
+
+
+def make_spec_tensor(ds, AUTOTUNE=None):
+    ds = ds.batch(1)
+    ds = ds.map(lambda x, *y: (spec()(x), *y), num_parallel_calls=AUTOTUNE)
+    return ds.unbatch()

acodet/train.py

@@ -0,0 +1,290 @@
+import os
+import time
+from pathlib import Path
+import numpy as np
+import tensorflow as tf
+import tensorflow_addons as tfa
+
+from acodet.funcs import save_model_results, get_train_set_size
+from acodet import models
+from acodet.plot_utils import plot_model_results, create_and_save_figure
+from acodet.tfrec import run_data_pipeline, prepare
+from acodet.augmentation import run_augment_pipeline
+from acodet import global_config as conf
+
+AUTOTUNE = tf.data.AUTOTUNE
+
+
+def run_training(
+    ModelClassName=conf.MODELCLASSNAME,
+    data_dir=conf.TFREC_DESTINATION,
+    # TODO trennen destination und standardpfad - oder doch nicht?
+    batch_size=conf.BATCH_SIZE,
+    epochs=conf.EPOCHS,
+    load_ckpt_path=conf.LOAD_CKPT_PATH,
+    load_g_ckpt=conf.LOAD_G_CKPT,
+    keras_mod_name=conf.KERAS_MOD_NAME,
+    steps_per_epoch=conf.STEPS_PER_EPOCH,
+    time_augs=conf.TIME_AUGS,
+    mixup_augs=conf.MIXUP_AUGS,
+    spec_aug=conf.SPEC_AUG,
+    data_description=conf.DATA_DESCRIPTION,
+    init_lr=conf.INIT_LR,
+    final_lr=conf.FINAL_LR,
+    pre_blocks=conf.PRE_BLOCKS,
+    f_score_beta=conf.F_SCORE_BETA,
+    f_score_thresh=conf.F_SCORE_THRESH,
+    unfreeze=conf.UNFREEZE,
+):
+    info_text = f"""Model run INFO:
+
+    model: untrained model 
+    dataset: {data_description}
+    comments: implemented proper on-the-fly augmentation
+
+    VARS:
+    data_path       = {data_dir}
+    batch_size      = {batch_size}
+    Model           = {ModelClassName}
+    keras_mod_name  = {keras_mod_name}
+    epochs          = {epochs}
+    load_ckpt_path  = {load_ckpt_path}
+    load_g_ckpt     = {load_g_ckpt}
+    steps_per_epoch = {steps_per_epoch}
+    f_score_beta    = {f_score_beta}
+    f_score_thresh  = {f_score_thresh}
+    bool_time_shift = {time_augs}
+    bool_MixUps     = {mixup_augs}
+    bool_SpecAug    = {spec_aug}
+    init_lr         = {init_lr}
+    final_lr        = {final_lr}
+    unfreeze       = {unfreeze}
+    preproc blocks  = {pre_blocks}
+    """
+
+    #############################################################################
+    #############################  RUN  #########################################
+    #############################################################################
+    data_dir = list(Path(data_dir).iterdir())
+
+    ########### INIT TRAINING RUN AND DIRECTORIES ###############################
+    time_start = time.strftime("%Y-%m-%d_%H", time.gmtime())
+    Path(f"../trainings/{time_start}").mkdir(exist_ok=True, parents=True)
+
+    n_train, n_noise = get_train_set_size(data_dir)
+    n_train_set = n_train * (
+        1 + time_augs + mixup_augs + spec_aug * 2
+    )  # // batch_size
+    print(
+        "Train set size = {}. Epoch should correspond to this amount of steps.".format(
+            n_train_set
+        ),
+        "\n",
+    )
+
+    seed = np.random.randint(100)
+    open(f"../trainings/{time_start}/training_info.txt", "w").write(info_text)
+
+    ###################### DATA PREPROC PIPELINE ################################
+
+    train_data = run_data_pipeline(
+        data_dir, data_dir="train", AUTOTUNE=AUTOTUNE
+    )
+    test_data = run_data_pipeline(data_dir, data_dir="test", AUTOTUNE=AUTOTUNE)
+    noise_data = run_data_pipeline(
+        data_dir, data_dir="noise", AUTOTUNE=AUTOTUNE
+    )
+
+    train_data = run_augment_pipeline(
+        train_data,
+        noise_data,
+        n_noise,
+        n_train,
+        time_augs,
+        mixup_augs,
+        seed,
+        spec_aug=spec_aug,
+        time_start=time_start,
+        plot=False,
+        random=False,
+    )
+    train_data = prepare(
+        train_data, batch_size, shuffle=True, shuffle_buffer=n_train_set * 3
+    )
+    if (
+        steps_per_epoch
+        and n_train_set // batch_size < epochs * steps_per_epoch
+    ):
+        train_data = train_data.repeat(
+            epochs * steps_per_epoch // (n_train_set // batch_size) + 1
+        )
+
+    test_data = prepare(test_data, batch_size)
+
+    #############################################################################
+    ######################### TRAINING ##########################################
+    #############################################################################
+
+    lr = tf.keras.optimizers.schedules.ExponentialDecay(
+        init_lr,
+        decay_steps=steps_per_epoch or n_train_set // batch_size,
+        decay_rate=(final_lr / init_lr) ** (1 / epochs),
+        staircase=True,
+    )
+
+    model = models.init_model(
+        model_instance=ModelClassName,
+        checkpoint_dir=f"../trainings/{load_ckpt_path}/unfreeze_no-TF",
+        keras_mod_name=keras_mod_name,
+        input_specs=True,
+    )
+
+    model.compile(
+        optimizer=tf.keras.optimizers.legacy.Adam(learning_rate=lr),
+        loss=tf.keras.losses.BinaryCrossentropy(),
+        metrics=[
+            tf.keras.metrics.BinaryAccuracy(),
+            tf.keras.metrics.Precision(),
+            tf.keras.metrics.Recall(),
+            tfa.metrics.FBetaScore(
+                num_classes=1,
+                beta=f_score_beta,
+                threshold=f_score_thresh,
+                name="fbeta",
+            ),
+            tfa.metrics.FBetaScore(
+                num_classes=1,
+                beta=1.0,
+                threshold=f_score_thresh,
+                name="fbeta1",
+            ),
+        ],
+    )
+
+    if unfreeze:
+        for layer in model.layers[pre_blocks:-unfreeze]:
+            layer.trainable = False
+
+    checkpoint_path = (
+        f"../trainings/{time_start}/unfreeze_{unfreeze}" + "/cp-last.ckpt"
+    )
+    checkpoint_dir = os.path.dirname(checkpoint_path)
+
+    cp_callback = tf.keras.callbacks.ModelCheckpoint(
+        filepath=checkpoint_path,
+        mode="min",
+        verbose=1,
+        save_weights_only=True,
+        save_freq="epoch",
+    )
+
+    model.save_weights(checkpoint_path)
+    hist = model.fit(
+        train_data,
+        epochs=epochs,
+        steps_per_epoch=steps_per_epoch,
+        validation_data=test_data,
+        callbacks=[cp_callback],
+    )
+    result = hist.history
+    save_model_results(checkpoint_dir, result)
+
+    ############## PLOT TRAINING PROGRESS & MODEL EVALUTAIONS ###################
+
+    plot_model_results(
+        time_start, data=data_description, init_lr=init_lr, final_lr=final_lr
+    )
+    ModelClass = getattr(models, ModelClassName)
+    create_and_save_figure(
+        ModelClass,
+        data_dir,
+        batch_size,
+        time_start,
+        plot_cm=True,
+        data=data_description,
+        keras_mod_name=keras_mod_name,
+    )
+
+
+def save_model(
+    string,
+    model,
+    lr=5e-4,
+    weight_clip=None,
+    f_score_beta=0.5,
+    f_score_thresh=0.5,
+):
+    model.compile(
+        optimizer=tf.keras.optimizers.Adam(learning_rate=lr),
+        loss=tf.keras.losses.BinaryCrossentropy(),
+        metrics=[
+            tf.keras.metrics.BinaryAccuracy(),
+            tf.keras.metrics.Precision(),
+            tf.keras.metrics.Recall(),
+            tfa.metrics.FBetaScore(
+                num_classes=1,
+                beta=f_score_beta,
+                threshold=f_score_thresh,
+                name="fbeta",
+            ),
+            tfa.metrics.FBetaScore(
+                num_classes=1,
+                beta=1.0,
+                threshold=f_score_thresh,
+                name="fbeta1",
+            ),
+        ],
+    )
+    model.save(f"acodet/src/models/{string}")
+
+
+##############################################################################
+##############################################################################
+####################### CONFIGURE TRAINING ###################################
+##############################################################################
+##############################################################################
+
+if __name__ == "__main__":
+    data_dir = list(Path(conf.TFREC_DESTINATION).iterdir())
+
+    epochs = [*[43] * 5, 100]
+    batch_size = [32] * 6
+    time_augs = [True]
+    mixup_augs = [True]
+    spec_aug = [True]
+    init_lr = [*[4e-4] * 5, 1e-5]
+    final_lr = [3e-6] * 6
+    weight_clip = [None] * 6
+    ModelClassName = ["GoogleMod"] * 6
+    keras_mod_name = [None] * 6
+    load_ckpt_path = [*[False] * 5, "2022-11-30_01"]
+    load_g_weights = [False]
+    steps_per_epoch = [1000]
+    data_description = [data_dir]
+    pre_blocks = [9]
+    f_score_beta = [0.5]
+    f_score_thresh = [0.5]
+    unfreeze = ["no-TF"]
+
+    for i in range(len(time_augs)):
+        run_training(
+            data_dir=data_dir,
+            epochs=epochs[i],
+            batch_size=batch_size[i],
+            time_augs=time_augs[i],
+            mixup_augs=mixup_augs[i],
+            spec_aug=spec_aug[i],
+            init_lr=init_lr[i],
+            final_lr=final_lr[i],
+            #  weight_clip=weight_clip[i],
+            ModelClassName=ModelClassName[i],
+            keras_mod_name=keras_mod_name[i],
+            load_ckpt_path=load_ckpt_path[i],
+            #  load_g_weights=load_g_weights[i],
+            steps_per_epoch=steps_per_epoch[i],
+            data_description=data_description[i],
+            pre_blocks=pre_blocks[i],
+            f_score_beta=f_score_beta[i],
+            f_score_thresh=f_score_thresh[i],
+            unfreeze=unfreeze[i],
+        )

advanced_config.yml

@@ -0,0 +1,106 @@
+##############################################################################
+
+# THIS FILE IS ONLY MEANT TO BE EDITED IF YOU ARE SURE!
+
+# PROGRAM FAILURE IS LIKELY TO OCCUR IF YOU ARE UNSURE OF THE
+# CONSEQUENCES OF YOUR CHANGES.
+
+# IF YOU HAVE CHANGED VALUES AND ARE ENCOUNTERING ERRORS, 
+# STASH YOUR CHANGES ('git stash' in a git bash console in acodet directory)
+
+##############################################################################
+    
+
+####################  AUDIO PROCESSING PARAMETERS  ###########################
+#!!! CHANGING THESE PARAMETERS WILL RESULT IN MODEL FAILURE, ONLY TO      !!!#
+#!!! BE CHANGED FOR TRAINING OF NEW MODEL ESPECIALY FOR DIFFERENT SPECIES !!!#
+## Global audio processing parameters
+# sample rate
+sample_rate: 2000
+# length of context window in seconds 
+# this is the audio segment length that the model is trained on
+context_window_in_seconds: 3.9
+
+## Mel-Spectrogram parameters
+# FFT window length
+stft_frame_len: 1024
+# number of time bins for mel spectrogram
+number_of_time_bins: 128
+
+
+###################  TFRECORD CREATION PARAMETERS  ###########################
+## Settings for Creation of Tfrecord Dataset
+# limit of context windows in a tfrecords file
+tfrecs_limit_per_file: 600
+# train/test split
+train_ratio: 0.7
+# test/val split
+test_val_ratio: 0.7
+
+########################  ANNOTATIONS ########################################
+default_threshold: 0.5
+# minimum frequency of annotation boxes
+annotation_df_fmin: 50
+# maximum frequency of annotation boxes
+annotation_df_fmax: 1000
+
+########################  PATHS  #############################################
+# dataset destination directory - save tfrecord dataset to this directory 
+# only change when really necessary
+tfrecords_destination_folder: '../Data/Datasets'
+# default folder to store newly created annotations
+generated_annotations_folder: '../generated_annotations'
+# name of current North Atlantic humpback whale song model
+model_name: 'Humpback_20221130'
+# name of top level directory when annotating multiple datasets
+top_dir_name: 'main'
+# custom string to add to timestamp for directory name 
+# of created annotations
+annots_timestamp_folder: ''
+# default threshold folder name
+thresh_label: 'thresh_0.5'
+
+#######################  TRAINING  ###########################################
+
+# Name of Model class, default is HumpBackNorthAtlantic, possible other classes
+# are GoogleMod for the modified ResNet-50 architecture, or KerasAppModel for 
+# any of the Keras application models (name will get specified under keras_mod_name)
+ModelClassName: 'HumpBackNorthAtlantic' 
+# batch size for training and evaluating purposes
+batch_size: 32
+# number of epochs to run the model
+epochs: 50
+# specify the path to your training checkpoint here to load a pretrained model
+load_ckpt_path: False
+# to run the google model, select True
+load_g_ckpt: False
+# specify the name of the keras application model that you want to run - select the
+# ModelClassName KerasAppModel for this
+keras_mod_name: False
+# number of steps per epoch
+steps_per_epoch: 1000
+# select True if you want your training data to be time shift augmented (recommended)
+time_augs: True 
+# select True if you want your training data to be MixUp augmented (recommended)
+mixup_augs: True
+# select True if you want your training data to be 
+# time and frequency masked augmented (recommended)
+spec_aug: True
+# specify a string to describe the dataset used for this model run (to later be able
+# to understand what was significant about this model training)
+data_description: 'describe your dataset'
+# starting learning rate
+init_lr: 5e-4
+# final learning rate
+final_lr: 5e-6
+# number of preliminary blocks in model (are kept frozen)
+pre_blocks: 9
+# threshold for f score beta
+f_score_beta: 0.5
+f_score_thresh: 0.5
+# number of layers to unfreeze, if False, the entire model is trainable
+unfreeze: False
+
+########################  Streamlit  #########################################
+# select True if you want to run the streamlit app
+streamlit: False

macM1_requirements/requirements_m1-1.txt

@@ -0,0 +1,11 @@
+keras-cv==0.5.0
+librosa==0.9.1
+matplotlib==3.5.1
+numpy==1.22.0
+pandas==1.4.2
+PyYAML==6.0.1
+seaborn==0.12.1
+streamlit==1.25.0
+tensorflow==2.13.0rc0
+tensorflow_io==0.31.0
+tensorflow-addons==0.20.0

macM1_requirements/requirements_m1-2.txt

@@ -0,0 +1 @@
+keras==2.12

pyproject.toml

@@ -0,0 +1,3 @@
+[tool.black]
+line-length = 79
+include = '\.pyi?$'

requirements.txt

@@ -0,0 +1,16 @@
+keras_cv==0.5.0
+librosa==0.9.1
+matplotlib==3.5.1
+numpy==1.22.0
+pandas==1.4.2
+PyYAML==6.0.1
+seaborn==0.12.2
+streamlit==1.25.0
+tensorflow==2.12.0
+tensorflow_addons==0.20.0
+tensorflow_intel
+tensorflow_io==0.31.0
+plotly==5.16.1
+black==23.7.0
+pre-commit==3.3.3
+pyqt5==5.15.6

run.py

@@ -0,0 +1,80 @@
+def main(sc=True, **kwargs):
+    """
+    Main function to run the whole pipeline. The function is called from the
+    streamlit app. The function is called with the preset option as an argument.
+    The preset option is used to determine which function should be called.
+    The preset option is set in the config file.
+
+    Parameters
+    ----------
+    sc : bool, optional
+        Decide if meta analysis should be done using sequence limit, by default True
+
+    Returns
+    -------
+    str or None
+        depending on the preset option, the function returns either the time
+        when the annotation was started or None
+    """
+    from acodet.annotate import run_annotation, filter_annots_by_thresh
+    from acodet.train import run_training, save_model
+    from acodet.tfrec import write_tfrec_dataset
+    from acodet.hourly_presence import compute_hourly_pres, calc_val_diff
+    from acodet.evaluate import create_overview_plot
+    from acodet.combine_annotations import generate_final_annotations
+    from acodet.models import init_model
+    import acodet.global_config as conf
+
+    if "fetch_config_again" in kwargs:
+        import importlib
+
+        importlib.reload(conf)
+        kwargs["relativ_path"] = conf.SOUND_FILES_SOURCE
+    if "preset" in kwargs:
+        preset = kwargs["preset"]
+    else:
+        preset = conf.PRESET
+
+    if conf.RUN_CONFIG == 1:
+        if preset == 1:
+            timestamp_foldername = run_annotation(**kwargs)
+            return timestamp_foldername
+        elif preset == 2:
+            new_thresh = filter_annots_by_thresh(**kwargs)
+            return new_thresh
+        elif preset == 3:
+            compute_hourly_pres(sc=sc, **kwargs)
+        elif preset == 4:
+            compute_hourly_pres(**kwargs)
+        elif preset == 6:
+            calc_val_diff(**kwargs)
+        elif preset == 0:
+            timestamp_foldername = run_annotation(**kwargs)
+            filter_annots_by_thresh(timestamp_foldername, **kwargs)
+            compute_hourly_pres(timestamp_foldername, sc=sc, **kwargs)
+            return timestamp_foldername
+
+    elif conf.RUN_CONFIG == 2:
+        if preset == 1:
+            generate_final_annotations(**kwargs)
+            write_tfrec_dataset(**kwargs)
+        elif preset == 2:
+            generate_final_annotations(active_learning=False, **kwargs)
+            write_tfrec_dataset(active_learning=False, **kwargs)
+
+    elif conf.RUN_CONFIG == 3:
+        if preset == 1:
+            run_training(**kwargs)
+        elif preset == 2:
+            create_overview_plot(**kwargs)
+        elif preset == 3:
+            create_overview_plot("2022-05-00_00", **kwargs)
+        elif preset == 4:
+            save_model("FlatHBNA", init_model(), **kwargs)
+
+
+if __name__ == "__main__":
+    from acodet.create_session_file import create_session_file
+
+    create_session_file()
+    main()

simple_config.yml

@@ -0,0 +1,99 @@
+##############################################################################
+
+# This file is for you to edit the paths corresponding to the following:
+# - source path of annotation files
+# - destination path of your annotation files (leave unchanged if possible)
+# - source path of sound files (.wav of .aif) (top most directory)
+# - destination path of any plots or spreadsheets
+
+# This file is also for you to edit the threshold value of the detector, to 
+# make the detector more sensitive or less sensitive.
+#   - Higher threshold will decrease number of false positives but
+#   at the cost of overlooking vocalizations.
+#   - Lower threshold values will increase number of false positives
+#   but more likely also detect false positives.
+
+
+##############################################################################
+
+
+
+
+###################### 1. DEFINE YOUR RUN ####################################
+
+# what would you like to do?
+# chose the run configuration: 
+#     - 1 generate annotations
+#     - 2 generate new training data from reviewed annotations
+#     - 3 train (and evaluate)
+run_config: 1
+
+# depending on the main task, chose your predefined settings:
+# for generation of annotations, chose:
+#     - 1 generate new annotations
+#     - 2 filter existing annotations with different threshold
+#     - 3 generate hourly predictions (simple limit and sequence criterion)
+#     - 4 generate hourly predictions (only simple limit)
+#     - 5 generate hourly predictions with varying limits - n.i.
+#     - 0 all of the above
+# for generation of new training data, chose:
+#     - 1 generate new training data from reviewed annotations
+#     - 2 generate new training data from reviewed annotations 
+#       and fill space between annotations with noise annotations
+# for training, chose:
+#     - 1 continue training on existing model and save model in the end
+#     - 2 evaluate saved model
+#     - 3 evaluate model checkpoint
+#     - 4 save model specified in advanced config
+predefined_settings: 0
+
+####################### 2. DEFINCE YOUR PATHS  ###############################
+
+## Paths
+# source path for your sound files (top most directory)
+# relevant for generation of new annotations (option run_config: 1)
+sound_files_source: 'path to your sound files'
+
+# source path for automatically generated annotations
+# relevant for generation of hourly or daily presence, or recomputing
+# with a different threshold
+# (options run_config: 1 and predifined_settings: 2 and 4)
+generated_annotation_source: '../Cathy/HBW_detector'
+
+
+# source path for annotations created or reviewed by you
+# relevant for creation of new dataset (option run_config: 2)
+#   -> might be easier to just copy annotation files to default location
+reviewed_annotation_source: '../annotations'
+
+# source path for automatically generated combined annotations 
+# Only relevant for creation of new dataset (option run_config: 2)
+# if unsure, leave unchanged. 
+annotation_destination: '../combined_annotations'
+
+
+
+
+####################### 3. DEFINE YOUR PARAMETERS  ###########################
+
+## Model Parameters
+# threshold for predictions
+thresh: 0.9
+# number of annotations above threshold for hourly presence (default = 15)
+simple_limit: 15
+# threshold for sequence criterion (default = 0.9)
+sequence_thresh: 0.9
+# number of annotations above threshold within 20 consecutive windows
+# for hourly presence (default = 3)
+sequence_limit: 3
+# number of consecutive windows that sc_limit has to occur in (default = 20)
+sequence_con_win: 20
+# number of annotations that correspond to the upper limit of color bar in 
+# hourly annotations plots
+max_annots_per_hour: 150
+# path to validation file of hourly presence dataset
+hourly_presence_validation_path: 'validation.csv'
+
+# number of predictions that get computed at once - should be in [20, 2000]
+# worth testing out different values to find whats fastest for your machine
+prediction_window_limit: 1000

streamlit_app.py

@@ -0,0 +1,94 @@
+import streamlit as st
+from acodet.create_session_file import create_session_file, read_session_file
+from acodet.front_end import help_strings
+
+if not "session_started" in st.session_state:
+    st.session_state.session_started = True
+    create_session_file()
+from acodet.front_end import (
+    utils,
+    st_annotate,
+    st_generate_data,
+    st_train,
+    st_visualization,
+)
+
+utils.write_to_session_file("streamlit", True)
+
+
+def select_preset():
+    utils.write_to_session_file("run_config", st.session_state.run_option)
+    show_run_btn = False
+
+    if st.session_state.run_option == 1:
+        show_run_btn = st_annotate.annotate_options()
+    elif st.session_state.run_option == 2:
+        show_run_btn = st_generate_data.generate_data_options()
+    elif st.session_state.run_option == 3:
+        show_run_btn = st_train.train_options()
+    if show_run_btn:
+        run_computions()
+
+
+def run_computions(**kwargs):
+    utils.next_button(id=4, text="Run computations")
+    if st.session_state.b4:
+        display_not_implemented_text()
+        kwargs = utils.prepare_run()
+        if not st.session_state.run_finished:
+            import run
+
+            st.session_state.save_dir = run.main(
+                fetch_config_again=True, **kwargs
+            )
+            st.session_state.run_finished = True
+
+    if st.session_state.run_finished:
+        if not st.session_state.preset_option == 3:
+            st.write("Computation finished")
+            utils.next_button(id=5, text="Show results")
+            st.markdown("""---""")
+        else:
+            conf = read_session_file()
+            st.session_state.b5 = True
+            st.session_state.save_dir = conf["generated_annotation_source"]
+
+        if not st.session_state.b5:
+            pass
+        else:
+            st_visualization.output()
+            st.stop()
+
+
+def display_not_implemented_text():
+    if not st.session_state.run_option == 1:
+        st.write(
+            """This option is not yet implemented for usage
+                    with the user interface. A headless version is
+                    available at https://github.com/vskode/acodet."""
+        )
+        st.stop()
+
+
+if __name__ == "__main__":
+    
+    st.markdown(
+        """
+        # Welcome to AcoDet - Acoustic Detection of Animal Vocalizations :loud_sound:
+        ### This program is currently equipped with a humpback whale song detector for the North Atlantic :whale2:
+        For more information, please visit https://github.com/vskode/acodet
+        
+        ---
+        """
+    )
+    run_option = int(
+        st.selectbox(
+            "How would you like run the program?",
+            ("1 - Inference", "2 - Generate new training data", "3 - Train"),
+            key="main",
+            help=help_strings.RUN_OPTION,
+        )[0]
+    )
+
+    st.session_state.run_option = run_option
+    select_preset()

tests/test.py

@@ -0,0 +1,132 @@
+import os, sys
+import unittest
+from pathlib import Path
+import pandas as pd
+
+sys.path.insert(0, os.path.abspath("."))
+
+########### MODIFY SESSION SETTINGS BEFORE GLOBAL CONFIG IS IMPORTED #########
+from acodet.create_session_file import create_session_file
+
+create_session_file()
+import json
+
+with open("acodet/src/tmp_session.json", "r") as f:
+    session = json.load(f)
+session["sound_files_source"] = "tests/test_files/test_audio_files"
+session[
+    "generated_annotation_source"
+] = "tests/test_files/test_generated_annotations"
+session[
+    "annotation_destination"
+] = "tests/test_files/test_combined_annotations"
+session[
+    "generated_annotations_folder"
+] = "tests/test_files/test_generated_annotations"
+
+session[
+    "reviewed_annotation_source"
+] = "tests/test_files/test_generated_annotations"
+session["tfrecords_destination_folder"] = "tests/test_files/test_tfrecords"
+
+with open("acodet/src/tmp_session.json", "w") as f:
+    json.dump(session, f)
+##############################################################################
+
+
+from acodet.annotate import run_annotation, filter_annots_by_thresh
+from acodet.funcs import return_windowed_file, get_train_set_size
+from acodet.models import GoogleMod
+from acodet.combine_annotations import generate_final_annotations
+from acodet.tfrec import write_tfrec_dataset
+from acodet.train import run_training
+from acodet import global_config as conf
+
+
+
+class TestDetection(unittest.TestCase):
+    def test_annotation(self):
+        self.time_stamp = run_annotation()
+        df = pd.read_csv(
+            (
+                Path(conf.GEN_ANNOTS_DIR)
+                .joinpath(self.time_stamp)
+                .joinpath("stats.csv")
+            )
+        )
+        self.assertEqual(
+            df["number of predictions with thresh>0.8"][0],
+            326,
+            "Number of predictions is not what it should be.",
+        )
+
+        filter_annots_by_thresh(self.time_stamp)
+        file = list(
+            Path(conf.GEN_ANNOT_SRC)
+            .joinpath(self.time_stamp)
+            .joinpath(f"thresh_{conf.THRESH}")
+            .glob("**/*.txt")
+        )[0]
+        df = pd.read_csv(file)
+        self.assertEqual(
+            len(df),
+            309,
+            "Number of predictions from filtered thresholds " "is incorrect.",
+        )
+
+
+class TestTraining(unittest.TestCase):
+    def test_model_load(self):
+        model = GoogleMod(load_g_ckpt=False).model
+        self.assertGreater(len(model.layers), 15)
+
+    # def test_tfrecord_loading(self):
+    #     data_dir = list(Path(conf.TFREC_DESTINATION).iterdir())
+    #     n_train, n_noise = get_train_set_size(data_dir)
+    #     self.assertEqual(n_train, 517)
+    #     self.assertEqual(n_noise, 42)
+
+class TestTFRecordCreation(unittest.TestCase):
+    def test_tfrecord(self):
+        time_stamp = list(Path(conf.ANNOT_DEST).iterdir())[-1]
+        write_tfrec_dataset(annot_dir=time_stamp, active_learning=False)
+        metadata_file_path = Path(conf.TFREC_DESTINATION).joinpath(
+            "dataset_meta_train.json"
+        )
+        self.assertEqual(
+            metadata_file_path.exists(),
+            1,
+            "TFRecords metadata file was not created.",
+        )
+
+        with open(metadata_file_path, "r") as f:
+            data = json.load(f)
+            self.assertEqual(
+                data["dataset"]["size"]["train"],
+                517,
+                "TFRecords files has wrong number of datapoints.",
+            )
+
+    def test_combined_annotation(self):
+        generate_final_annotations(active_learning=False)
+        time_stamp = list(Path(conf.GEN_ANNOTS_DIR).iterdir())[-1].stem
+        combined_annots_path = (
+            Path(conf.ANNOT_DEST)
+            .joinpath(time_stamp)
+            .joinpath("combined_annotations.csv")
+        )
+        self.assertEqual(
+            combined_annots_path.exists(),
+            1,
+            "csv file containing combined_annotations does not exist.",
+        )
+        df = pd.read_csv(combined_annots_path)
+        self.assertEqual(
+            df.start.iloc[-1],
+            1795.2825,
+            "The annotations in combined_annotations.csv don't seem to be identical",
+        )
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/test_files/test_annoted_files/thresh_0.5/N1/2021-03-10_2kHz/channelA_2021-03-10_00-00-05_annot_2022-11-30_01.txt

@@ -0,0 +1,28 @@
+Selection	Begin Time (s)	End Time (s)	High Freq (Hz)	Low Freq (Hz)	Prediction/Comments
+1	3.8775	7.755	1000	50	0.72177804
+2	65.9175	69.795	1000	50	0.5281566
+3	108.57	112.44749999999999	1000	50	0.76347905
+4	124.08	127.9575	1000	50	0.64476347
+5	127.9575	131.835	1000	50	0.6341658
+6	139.59	143.4675	1000	50	0.60823405
+7	178.365	182.2425	1000	50	0.6451596
+8	186.12	189.9975	1000	50	0.82998466
+9	201.63	205.5075	1000	50	0.75589406
+10	213.2625	217.14	1000	50	0.6491813
+11	217.14	221.01749999999998	1000	50	0.7016125
+12	275.3025	279.18	1000	50	0.8662185
+13	290.8125	294.69	1000	50	0.5251316
+14	321.8325	325.71	1000	50	0.64809465
+15	407.1375	411.015	1000	50	0.5973742
+16	411.015	414.8925	1000	50	0.9908187
+17	476.9325	480.81	1000	50	0.85746086
+18	480.81	484.6875	1000	50	0.71086293
+19	507.9525	511.83	1000	50	0.79394937
+20	511.83	515.7075	1000	50	0.91436124
+21	546.7275	550.605	1000	50	0.6516418
+22	601.0125	604.8900000000001	1000	50	0.5256015
+23	616.5225	620.4000000000001	1000	50	0.8155822
+24	647.5425	651.4200000000001	1000	50	0.6215045
+25	659.175	663.0525	1000	50	0.7368213
+26	713.46	717.3375000000001	1000	50	0.57750905
+27	725.0925	728.97	1000	50	0.6525679

tests/test_files/test_annoted_files/thresh_0.5/N1/2021-03-10_2kHz/channelA_2021-03-10_00-15-55_annot_2022-11-30_01.txt

@@ -0,0 +1,20 @@
+Selection	Begin Time (s)	End Time (s)	High Freq (Hz)	Low Freq (Hz)	Prediction/Comments
+1	3.8775	7.755	1000	50	0.5005236
+2	50.4075	54.285	1000	50	0.50911117
+3	139.59	143.4675	1000	50	0.59164804
+4	162.855	166.7325	1000	50	0.54786545
+5	178.365	182.2425	1000	50	0.53832465
+6	201.63	205.5075	1000	50	0.759787
+7	213.2625	217.14	1000	50	0.7325302
+8	232.65	236.5275	1000	50	0.73861456
+9	244.2825	248.16	1000	50	0.5488199
+10	248.16	252.0375	1000	50	0.5221725
+11	275.3025	279.18	1000	50	0.604062
+12	290.8125	294.69	1000	50	0.54666525
+13	306.3225	310.2	1000	50	0.78042346
+14	414.8925	418.77	1000	50	0.61935645
+15	430.4025	434.28	1000	50	0.5925473
+16	445.9125	449.79	1000	50	0.57290447
+17	476.9325	480.81	1000	50	0.57334906
+18	496.32	500.1975	1000	50	0.75015646
+19	523.4625	527.34	1000	50	0.50864077

tests/test_files/test_annoted_files/thresh_0.5/N1/2021-03-10_2kHz/channelA_2021-03-10_01-00-05_annot_2022-11-30_01.txt

@@ -0,0 +1,25 @@
+Selection	Begin Time (s)	End Time (s)	High Freq (Hz)	Low Freq (Hz)	Prediction/Comments
+1	34.8975	38.775	1000	50	0.6350528
+2	89.1825	93.06	1000	50	0.7238582
+3	151.2225	155.1	1000	50	0.7220111
+4	217.14	221.01749999999998	1000	50	0.59247476
+5	221.0175	224.895	1000	50	0.58241534
+6	232.65	236.5275	1000	50	0.9417606
+7	244.2825	248.16	1000	50	0.5242218
+8	279.18	283.0575	1000	50	0.50423527
+9	294.69	298.5675	1000	50	0.5808482
+10	337.3425	341.21999999999997	1000	50	0.5212356
+11	352.8525	356.73	1000	50	0.7410249
+12	356.73	360.6075	1000	50	0.8129122
+13	372.24	376.1175	1000	50	0.55076885
+14	403.26	407.1375	1000	50	0.5803523
+15	457.545	461.4225	1000	50	0.5226226
+16	527.34	531.2175000000001	1000	50	0.69937265
+17	569.9925	573.87	1000	50	0.53878176
+18	682.44	686.3175000000001	1000	50	0.5675298
+19	725.0925	728.97	1000	50	0.66011316
+20	767.745	771.6225000000001	1000	50	0.8282536
+21	969.375	973.2525	1000	50	0.53780836
+22	981.0075	984.8850000000001	1000	50	0.714791
+23	1019.7825	1023.6600000000001	1000	50	0.65036213
+24	1031.415	1035.2925	1000	50	0.58890146

tests/test_files/test_annoted_files/thresh_0.5/N1/2021-03-10_2kHz/channelA_2021-03-10_01-21-14_annot_2022-11-30_01.txt

@@ -0,0 +1,6 @@
+Selection	Begin Time (s)	End Time (s)	High Freq (Hz)	Low Freq (Hz)	Prediction/Comments
+1	34.8975	38.775	1000	50	0.5121363
+2	69.795	73.6725	1000	50	0.8324235
+3	135.7125	139.59	1000	50	0.5703977
+4	201.63	205.5075	1000	50	0.94431615
+5	224.895	228.7725	1000	50	0.661064

tests/test_files/test_annoted_files/thresh_0.5/N1/2021-03-10_2kHz/channelA_2021-03-10_02-00-05_annot_2022-11-30_01.txt

@@ -0,0 +1,60 @@
+Selection	Begin Time (s)	End Time (s)	High Freq (Hz)	Low Freq (Hz)	Prediction/Comments
+1	38.775	42.652499999999996	1000	50	0.737386
+2	100.815	104.6925	1000	50	0.72394115
+3	108.57	112.44749999999999	1000	50	0.5832078
+4	116.325	120.2025	1000	50	0.5326732
+5	139.59	143.4675	1000	50	0.6070674
+6	143.4675	147.345	1000	50	0.868954
+7	174.4875	178.365	1000	50	0.8059232
+8	213.2625	217.14	1000	50	0.6078309
+9	224.895	228.7725	1000	50	0.6188046
+10	275.3025	279.18	1000	50	0.7676495
+11	286.935	290.8125	1000	50	0.63999677
+12	294.69	298.5675	1000	50	0.69478
+13	306.3225	310.2	1000	50	0.5553063
+14	314.0775	317.955	1000	50	0.5727619
+15	356.73	360.6075	1000	50	0.5218362
+16	395.505	399.3825	1000	50	0.5227227
+17	399.3825	403.26	1000	50	0.79591954
+18	411.015	414.8925	1000	50	0.69747454
+19	422.6475	426.525	1000	50	0.5296574
+20	438.1575	442.035	1000	50	0.5216314
+21	442.035	445.9125	1000	50	0.80506706
+22	515.7075	519.585	1000	50	0.72778624
+23	531.2175	535.095	1000	50	0.6537169
+24	535.095	538.9725000000001	1000	50	0.6939367
+25	566.115	569.9925000000001	1000	50	0.54583424
+26	612.645	616.5225	1000	50	0.5596068
+27	635.91	639.7875	1000	50	0.87055826
+28	647.5425	651.4200000000001	1000	50	0.5198072
+29	690.195	694.0725000000001	1000	50	0.6586496
+30	697.95	701.8275000000001	1000	50	0.59695214
+31	752.235	756.1125000000001	1000	50	0.5564991
+32	759.99	763.8675000000001	1000	50	0.8687321
+33	771.6225	775.5	1000	50	0.745041
+34	794.8875	798.7650000000001	1000	50	0.63954085
+35	829.785	833.6625	1000	50	0.58957016
+36	841.4175	845.2950000000001	1000	50	0.70007896
+37	845.295	849.1725	1000	50	0.5040522
+38	856.9275	860.8050000000001	1000	50	0.50218743
+39	899.58	903.4575000000001	1000	50	0.64664465
+40	926.7225	930.6	1000	50	0.61015755
+41	934.4775	938.355	1000	50	0.6043967
+42	965.4975	969.375	1000	50	0.592605
+43	1004.2725	1008.1500000000001	1000	50	0.5635989
+44	1019.7825	1023.6600000000001	1000	50	0.7166601
+45	1023.66	1027.5375	1000	50	0.603185
+46	1043.0475	1046.925	1000	50	0.6931752
+47	1077.945	1081.8225	1000	50	0.64394623
+48	1089.5775	1093.4550000000002	1000	50	0.54892826
+49	1159.3725	1163.25	1000	50	0.65110403
+50	1209.78	1213.6575	1000	50	0.52652115
+51	1213.6575	1217.535	1000	50	0.59059477
+52	1217.535	1221.4125000000001	1000	50	0.53762406
+53	1233.045	1236.9225000000001	1000	50	0.77236694
+54	1267.9425	1271.8200000000002	1000	50	0.65751004
+55	1279.575	1283.4525	1000	50	0.8536242
+56	1318.35	1322.2275	1000	50	0.7538498
+57	1349.37	1353.2475	1000	50	0.57182205
+58	1361.0025	1364.88	1000	50	0.6505617
+59	1368.7575	1372.635	1000	50	0.64853406

tests/test_files/test_annoted_files/thresh_0.5/N1/2021-03-10_2kHz/channelA_2021-03-10_03-00-05_annot_2022-11-30_01.txt

@@ -0,0 +1,83 @@
+Selection	Begin Time (s)	End Time (s)	High Freq (Hz)	Low Freq (Hz)	Prediction/Comments
+1	7.755	11.6325	1000	50	0.75583506
+2	15.51	19.3875	1000	50	0.5251516
+3	27.1425	31.02	1000	50	0.55041605
+4	31.02	34.8975	1000	50	0.85942334
+5	42.6525	46.53	1000	50	0.9165694
+6	46.53	50.4075	1000	50	0.7389612
+7	50.4075	54.285	1000	50	0.8961259
+8	58.1625	62.04	1000	50	0.58524656
+9	69.795	73.6725	1000	50	0.7369542
+10	77.55	81.4275	1000	50	0.73338825
+11	155.1	158.9775	1000	50	0.63714325
+12	209.385	213.2625	1000	50	0.57494605
+13	259.7925	263.67	1000	50	0.77267367
+14	263.67	267.5475	1000	50	0.5608314
+15	275.3025	279.18	1000	50	0.56708634
+16	286.935	290.8125	1000	50	0.53724974
+17	310.2	314.0775	1000	50	0.6503988
+18	376.1175	379.995	1000	50	0.5667686
+19	379.995	383.8725	1000	50	0.6167431
+20	387.75	391.6275	1000	50	0.69172144
+21	395.505	399.3825	1000	50	0.6661956
+22	399.3825	403.26	1000	50	0.6705925
+23	449.79	453.6675	1000	50	0.76984024
+24	496.32	500.1975	1000	50	0.6549141
+25	535.095	538.9725000000001	1000	50	0.7988651
+26	546.7275	550.605	1000	50	0.98777056
+27	558.36	562.2375000000001	1000	50	0.5757415
+28	581.625	585.5025	1000	50	0.9520807
+29	608.7675	612.6450000000001	1000	50	0.7059972
+30	651.42	655.2975	1000	50	0.58196646
+31	659.175	663.0525	1000	50	0.50375336
+32	663.0525	666.9300000000001	1000	50	0.7096761
+33	666.93	670.8075	1000	50	0.6709009
+34	674.685	678.5625	1000	50	0.6068424
+35	705.705	709.5825000000001	1000	50	0.79320824
+36	709.5825	713.46	1000	50	0.7026795
+37	717.3375	721.215	1000	50	0.59928995
+38	721.215	725.0925000000001	1000	50	0.8359426
+39	736.725	740.6025000000001	1000	50	0.52542675
+40	740.6025	744.48	1000	50	0.56663805
+41	756.1125	759.99	1000	50	0.51278627
+42	775.5	779.3775	1000	50	0.5195257
+43	779.3775	783.2550000000001	1000	50	0.9383165
+44	806.52	810.3975	1000	50	0.74511904
+45	814.275	818.1525	1000	50	0.72356
+46	841.4175	845.2950000000001	1000	50	0.50421375
+47	853.05	856.9275	1000	50	0.57719857
+48	868.56	872.4375	1000	50	0.5337666
+49	903.4575	907.335	1000	50	0.6433379
+50	918.9675	922.845	1000	50	0.7503527
+51	934.4775	938.355	1000	50	0.5536823
+52	946.11	949.9875000000001	1000	50	0.79101586
+53	957.7425	961.62	1000	50	0.6153837
+54	965.4975	969.375	1000	50	0.63526756
+55	969.375	973.2525	1000	50	0.6462273
+56	973.2525	977.1300000000001	1000	50	0.6215711
+57	984.885	988.7625	1000	50	0.5670401
+58	1012.0275	1015.9050000000001	1000	50	0.6619715
+59	1015.905	1019.7825	1000	50	0.6685599
+60	1039.17	1043.0475000000001	1000	50	0.755065
+61	1046.925	1050.8025	1000	50	0.51247126
+62	1050.8025	1054.68	1000	50	0.7799243
+63	1074.0675	1077.9450000000002	1000	50	0.798874
+64	1136.1075	1139.9850000000001	1000	50	0.8805855
+65	1143.8625	1147.74	1000	50	0.50031465
+66	1182.6375	1186.515	1000	50	0.6415266
+67	1190.3925	1194.27	1000	50	0.77716064
+68	1198.1475	1202.025	1000	50	0.6745678
+69	1209.78	1213.6575	1000	50	0.5046
+70	1217.535	1221.4125000000001	1000	50	0.51890004
+71	1310.595	1314.4725	1000	50	0.76617485
+72	1326.105	1329.9825	1000	50	0.5635491
+73	1337.7375	1341.615	1000	50	0.6052231
+74	1361.0025	1364.88	1000	50	0.6599168
+75	1364.88	1368.7575000000002	1000	50	0.6918338
+76	1368.7575	1372.635	1000	50	0.58110696
+77	1384.2675	1388.145	1000	50	0.80470777
+78	1388.145	1392.0225	1000	50	0.772092
+79	1399.7775	1403.655	1000	50	0.5838019
+80	1415.2875	1419.165	1000	50	0.5703714
+81	1450.185	1454.0625	1000	50	0.75559676
+82	1492.8375	1496.7150000000001	1000	50	0.5252385

tests/test_files/test_annoted_files/thresh_0.5/N1/2021-03-10_2kHz/channelA_2021-03-10_04-00-05_annot_2022-11-30_01.txt

@@ -0,0 +1,5 @@
+Selection	Begin Time (s)	End Time (s)	High Freq (Hz)	Low Freq (Hz)	Prediction/Comments
+1	0.0	3.8775	1000	50	0.7659342
+2	50.4075	54.285	1000	50	0.5111502
+3	73.6725	77.55	1000	50	0.5522041
+4	89.1825	93.06	1000	50	0.6636806

tests/test_files/test_annoted_files/thresh_0.5/N1/2021-03-10_2kHz/channelA_2021-03-10_04-05-20_annot_2022-11-30_01.txt

@@ -0,0 +1,75 @@
+Selection	Begin Time (s)	End Time (s)	High Freq (Hz)	Low Freq (Hz)	Prediction/Comments
+1	34.8975	38.775	1000	50	0.71619654
+2	46.53	50.4075	1000	50	0.55028105
+3	50.4075	54.285	1000	50	0.5685571
+4	62.04	65.9175	1000	50	0.5583923
+5	73.6725	77.55	1000	50	0.61653876
+6	100.815	104.6925	1000	50	0.5019687
+7	104.6925	108.57	1000	50	0.5498319
+8	112.4475	116.325	1000	50	0.523319
+9	147.345	151.2225	1000	50	0.5715398
+10	166.7325	170.60999999999999	1000	50	0.58472294
+11	221.0175	224.895	1000	50	0.6042095
+12	228.7725	232.65	1000	50	0.5261942
+13	279.18	283.0575	1000	50	0.6790548
+14	294.69	298.5675	1000	50	0.88658756
+15	302.445	306.3225	1000	50	0.5844379
+16	314.0775	317.955	1000	50	0.5711364
+17	325.71	329.5875	1000	50	0.672497
+18	337.3425	341.21999999999997	1000	50	0.6838482
+19	356.73	360.6075	1000	50	0.506336
+20	360.6075	364.485	1000	50	0.68574035
+21	403.26	407.1375	1000	50	0.51428175
+22	449.79	453.6675	1000	50	0.8651263
+23	465.3	469.1775	1000	50	0.5493179
+24	476.9325	480.81	1000	50	0.5709537
+25	507.9525	511.83	1000	50	0.54585946
+26	519.585	523.4625000000001	1000	50	0.5220416
+27	527.34	531.2175000000001	1000	50	0.5410059
+28	535.095	538.9725000000001	1000	50	0.72252417
+29	546.7275	550.605	1000	50	0.74286765
+30	550.605	554.4825000000001	1000	50	0.82613313
+31	554.4825	558.36	1000	50	0.5101495
+32	601.0125	604.8900000000001	1000	50	0.5304
+33	635.91	639.7875	1000	50	0.60296184
+34	639.7875	643.6650000000001	1000	50	0.5382861
+35	655.2975	659.1750000000001	1000	50	0.5297241
+36	666.93	670.8075	1000	50	0.621523
+37	686.3175	690.195	1000	50	0.75933427
+38	697.95	701.8275000000001	1000	50	0.7845717
+39	709.5825	713.46	1000	50	0.6805964
+40	736.725	740.6025000000001	1000	50	0.73043954
+41	744.48	748.3575000000001	1000	50	0.51064
+42	756.1125	759.99	1000	50	0.8632106
+43	763.8675	767.745	1000	50	0.5147054
+44	775.5	779.3775	1000	50	0.71989703
+45	779.3775	783.2550000000001	1000	50	0.50403214
+46	783.255	787.1325	1000	50	0.5747235
+47	806.52	810.3975	1000	50	0.7938242
+48	814.275	818.1525	1000	50	0.5156269
+49	818.1525	822.0300000000001	1000	50	0.6604109
+50	841.4175	845.2950000000001	1000	50	0.54860944
+51	845.295	849.1725	1000	50	0.5326691
+52	849.1725	853.0500000000001	1000	50	0.96912056
+53	860.805	864.6825	1000	50	0.5358853
+54	872.4375	876.315	1000	50	0.5902473
+55	922.845	926.7225000000001	1000	50	0.52094966
+56	930.6	934.4775000000001	1000	50	0.95947707
+57	942.2325	946.11	1000	50	0.5568109
+58	961.62	965.4975000000001	1000	50	0.6030713
+59	973.2525	977.1300000000001	1000	50	0.5722379
+60	992.64	996.5175	1000	50	0.60539764
+61	1046.925	1050.8025	1000	50	0.6535185
+62	1054.68	1058.5575000000001	1000	50	0.5753429
+63	1062.435	1066.3125	1000	50	0.6893821
+64	1074.0675	1077.9450000000002	1000	50	0.61925906
+65	1077.945	1081.8225	1000	50	0.6268375
+66	1081.8225	1085.7	1000	50	0.784415
+67	1085.7	1089.5775	1000	50	0.50539637
+68	1097.3325	1101.21	1000	50	0.76099944
+69	1139.985	1143.8625	1000	50	0.54786134
+70	1147.74	1151.6175	1000	50	0.538391
+71	1159.3725	1163.25	1000	50	0.5029607
+72	1167.1275	1171.005	1000	50	0.8406276
+73	1174.8825	1178.76	1000	50	0.5073458
+74	1178.76	1182.6375	1000	50	0.6157743

tests/test_files/test_annoted_files/thresh_0.5/N1/2021-03-10_2kHz/channelA_2021-03-10_05-00-05_annot_2022-11-30_01.txt

@@ -0,0 +1,44 @@
+Selection	Begin Time (s)	End Time (s)	High Freq (Hz)	Low Freq (Hz)	Prediction/Comments
+1	7.755	11.6325	1000	50	0.8519168
+2	19.3875	23.265	1000	50	0.7065482
+3	27.1425	31.02	1000	50	0.5228868
+4	31.02	34.8975	1000	50	0.66215074
+5	34.8975	38.775	1000	50	0.694213
+6	69.795	73.6725	1000	50	0.7757755
+7	77.55	81.4275	1000	50	0.7832104
+8	89.1825	93.06	1000	50	0.6566331
+9	93.06	96.9375	1000	50	0.58944553
+10	96.9375	100.815	1000	50	0.55878145
+11	104.6925	108.57	1000	50	0.7410829
+12	127.9575	131.835	1000	50	0.69165194
+13	139.59	143.4675	1000	50	0.79746383
+14	143.4675	147.345	1000	50	0.9918664
+15	166.7325	170.60999999999999	1000	50	0.989512
+16	170.61	174.4875	1000	50	0.6965571
+17	178.365	182.2425	1000	50	0.83577895
+18	189.9975	193.875	1000	50	0.51267475
+19	193.875	197.7525	1000	50	0.5540345
+20	205.5075	209.385	1000	50	0.6198408
+21	209.385	213.2625	1000	50	0.9075451
+22	228.7725	232.65	1000	50	0.7090942
+23	232.65	236.5275	1000	50	0.546338
+24	240.405	244.2825	1000	50	0.7075751
+25	244.2825	248.16	1000	50	0.92911464
+26	252.0375	255.915	1000	50	0.9545463
+27	255.915	259.7925	1000	50	0.87419856
+28	267.5475	271.425	1000	50	0.74750215
+29	271.425	275.3025	1000	50	0.7619019
+30	279.18	283.0575	1000	50	0.5040308
+31	286.935	290.8125	1000	50	0.6829701
+32	294.69	298.5675	1000	50	0.7550749
+33	310.2	314.0775	1000	50	0.53570014
+34	325.71	329.5875	1000	50	0.51625097
+35	337.3425	341.21999999999997	1000	50	0.558227
+36	352.8525	356.73	1000	50	0.542528
+37	356.73	360.6075	1000	50	0.732914
+38	360.6075	364.485	1000	50	0.6010917
+39	364.485	368.3625	1000	50	0.53385454
+40	391.6275	395.505	1000	50	0.7137319
+41	395.505	399.3825	1000	50	0.9997557
+42	399.3825	403.26	1000	50	0.95312613
+43	403.26	407.1375	1000	50	0.9119531

tests/test_files/test_annoted_files/thresh_0.5/N1/2021-03-10_2kHz/channelA_2021-03-10_05-10-39_annot_2022-11-30_01.txt

@@ -0,0 +1,73 @@
+Selection	Begin Time (s)	End Time (s)	High Freq (Hz)	Low Freq (Hz)	Prediction/Comments
+1	19.3875	23.265	1000	50	0.7046817
+2	62.04	65.9175	1000	50	0.89541245
+3	77.55	81.4275	1000	50	0.5546575
+4	81.4275	85.30499999999999	1000	50	0.67314935
+5	89.1825	93.06	1000	50	0.6299111
+6	93.06	96.9375	1000	50	0.9997956
+7	108.57	112.44749999999999	1000	50	0.56945264
+8	120.2025	124.08	1000	50	0.5600914
+9	124.08	127.9575	1000	50	0.99446577
+10	139.59	143.4675	1000	50	0.6193613
+11	143.4675	147.345	1000	50	0.51982135
+12	147.345	151.2225	1000	50	0.9659394
+13	162.855	166.7325	1000	50	0.9561064
+14	170.61	174.4875	1000	50	0.7320249
+15	197.7525	201.63	1000	50	0.9880733
+16	224.895	228.7725	1000	50	0.639691
+17	236.5275	240.405	1000	50	0.68663204
+18	244.2825	248.16	1000	50	0.5863455
+19	255.915	259.7925	1000	50	0.98858786
+20	263.67	267.5475	1000	50	0.5719061
+21	286.935	290.8125	1000	50	0.50176364
+22	290.8125	294.69	1000	50	0.71817106
+23	314.0775	317.955	1000	50	0.6869853
+24	325.71	329.5875	1000	50	0.5520363
+25	333.465	337.3425	1000	50	0.6743646
+26	368.3625	372.24	1000	50	0.508152
+27	383.8725	387.75	1000	50	0.76186216
+28	391.6275	395.505	1000	50	0.6509082
+29	395.505	399.3825	1000	50	0.5752171
+30	403.26	407.1375	1000	50	0.72370225
+31	461.4225	465.3	1000	50	0.50161684
+32	465.3	469.1775	1000	50	0.566452
+33	476.9325	480.81	1000	50	0.6763864
+34	488.565	492.4425	1000	50	0.5415367
+35	492.4425	496.32	1000	50	0.628716
+36	507.9525	511.83	1000	50	0.9948579
+37	515.7075	519.585	1000	50	0.98039323
+38	531.2175	535.095	1000	50	0.59561306
+39	535.095	538.9725000000001	1000	50	0.74321645
+40	538.9725	542.85	1000	50	0.7436308
+41	542.85	546.7275000000001	1000	50	0.7321441
+42	550.605	554.4825000000001	1000	50	0.6311483
+43	562.2375	566.115	1000	50	0.68326
+44	573.87	577.7475000000001	1000	50	0.7051835
+45	585.5025	589.3800000000001	1000	50	0.6416048
+46	589.38	593.2575	1000	50	0.8965987
+47	597.135	601.0125	1000	50	0.5137316
+48	604.89	608.7675	1000	50	0.6676253
+49	620.4	624.2775	1000	50	0.65750295
+50	624.2775	628.1550000000001	1000	50	0.76009196
+51	628.155	632.0325	1000	50	0.6924573
+52	639.7875	643.6650000000001	1000	50	0.91235334
+53	647.5425	651.4200000000001	1000	50	0.74130136
+54	651.42	655.2975	1000	50	0.5356798
+55	663.0525	666.9300000000001	1000	50	0.6730036
+56	666.93	670.8075	1000	50	0.78788316
+57	678.5625	682.44	1000	50	0.9130233
+58	701.8275	705.705	1000	50	0.7925505
+59	705.705	709.5825000000001	1000	50	0.67838407
+60	744.48	748.3575000000001	1000	50	0.6167371
+61	759.99	763.8675000000001	1000	50	0.59807014
+62	771.6225	775.5	1000	50	0.52345234
+63	787.1325	791.0100000000001	1000	50	0.7457069
+64	794.8875	798.7650000000001	1000	50	0.5075276
+65	798.765	802.6425	1000	50	0.6309759
+66	806.52	810.3975	1000	50	0.6761872
+67	814.275	818.1525	1000	50	0.786669
+68	818.1525	822.0300000000001	1000	50	0.5443038
+69	825.9075	829.7850000000001	1000	50	0.9998741
+70	837.54	841.4175	1000	50	0.8319586
+71	841.4175	845.2950000000001	1000	50	0.5379799
+72	849.1725	853.0500000000001	1000	50	0.6535889