Moon river

A (series of) Mysterious Flask Error(s)

Rose Lin -

The company-wise hack day for 2020 is tomorrow. It's my first time participating, so I'm a bit excited. We have an awesome idea to implement, which requires a simple Flask web app. Flashback to late 2018, I did write a simple Flask app by following this tutorial so that my hubby (or other interested parties) may upload cat photos to my external storage on Cloudinary. It worked, but uploading multiple photos always timed out. (Heroku seems to allow for max. 30 seconds idled time) Now that I contemplate this, I realize that I could have handled it through JS async call. But that's another story. In short, I'd like to reuse some legacy Flask codes that I somehow wrote in Python 2.7 (weird, right? Already late 2018, and the tutorial is based on Python 3, but I was still attached to Python 2?) as a boilerplate for my new project. I didn't imagine it would be that hard, though.

Import within the same folder no longer works?

Below is a snapshot of my folder structure:

+-- app
|   +-- __init__.py
|   +-- config.py
|   +-- ... (Other py files)
+-- migrations
|   +-- (Auto migration files created by SQLAlchemy)
+-- myapp.py
+-- requirements.txt

Within __init__.py, I have from config import Config. config.py is the configuration file with a Config class object. That's what the tutorial has, and works perfectly fine with Python 2.7.

After installing all the required packages, I specified the Flask app by setting export FLASK_APP=myapp.py, followed by flask run. The following error occurred:

File ".../app/__init__.py", line 7, in <module>
  from config import Config
ModuleNotFoundError: No module named 'config'

That's weird. I have a config.py resided in the same folder with __init__.py! How come it's not found?

Googled Solution 1: Out of sight, out of mind

The first search term that occurred in my mind was a relative import. (Spoiler alert: this turned out to be NOT related to relative import.) I quickly googled and found a solution:

If your config.py file and init.py file on the same path, move config.py file outside of the path and place the same in root.
for example if both are in “APP” directory, move the config.py and place it inside the root director of “APP” directory

Wow, sounds like exactly what I've been going through so far! Can't wait to try it!

After moving the config.py to the parent folder (aka. not within the app folder), I tried to invoke flask run again. This time, I got similar errors from importing other python files within the same app folder for the __init__.py file. There is no way I could move all files to the parent folder! Something is bound to be wrong in this case.

Googled Solution 2: Add system path

I dig a little deeper on Python 3 relative import and believed that this should be my cure:

I had a similar problem, I solved it by explicitly adding the file's directory to the path list:

import os
import sys

file_dir = os.path.dirname(__file__)
sys.path.append(file_dir)

After that, I had no problem importing from the same directory.

I copied and pasted the code to __init__.py and voila! Not only config.py could be seen, but also other files under the same app folder!

... But what about the local database?

The app has a SQLite local database that contains only one table to store registered user information. I was still not able to run the app due to the following error:

sqlalchemy.exc.InvalidRequestError: Table 'user' is already defined for this MetaData instance.  Specify 'extend_existing=True' to redefine options and columns on an existing Table object.

Per this suggestion I deleted all legacy .pyc files inherited from the repository. The error persisted. I tried this solution and it did an amazing job paving the path clear for me. Until I found out that I was not able to create new user anymore. In other words, the local database was corrupted, and I could not upgrade/downgrade it anymore.

Correct way: still around file imports.

I almost gave up after googling and trying different things for about an hour. Intensive coding is fun, but I would like to get the ball rolling ASAP, or it's better to start from scratch.

I prefer to edit codes directly from text editors such as Atom/Notepad++. But I do have IDEs such as PyCharm, which I love to work with when navigating projects with complex code structure. Somehow I found that PyCharm highlighted the row from config import Config with red squiggly lines. I removed the whole sys.path.append(file_dir)... block from __init__.py and tried one last thing: from app.config import Config. And... the red squiggly lines went away? Inspired by this finding, I immediately fixed all imports within the app folder by adding "app" in front of the filename. The problem is finally solved!

Afterthoughts

I wonder if it is because the app folder contains __init__.py thus Python 3.7 considered it as a package, therefore the "app" prefix is required to import the remaining files within the same folder. I vaguely recall myself reading a note on how frustrating it could be when dealing with Python import system. I probably should read it again.

TL;DR

  • A legacy Flask app could not run when migrating to another machine. It first complained that modules within the same folder cannot be found. Using some Googled trick I messed up the SQLite local database.
  • It's an issue with the Python import system.
  • I hate database migrations. (One main issue the team always ran into when I was a back-end intern for a web app)
  • Googled solutions do not always work; try at your own risk, but they may inspire you to find the right way.

And I hope I could bring some good news next time! Looks like only after dealing with a tricky tech problem do I have the impulse to update here...