Just How Easy Migrations Are With Redis

Published on 09/19/19 at 22:25 EST by Max Bridgland

I came to a conclusion that the home page of this blog had posts that were much too long. I needed to change it so that it would only display X characters and then a show more button linking to the blog post. Below is a migration script I wrote showing just how easy it was to convert my old db values to new ones.

def convert_home_html_to_new_format():
    home_post_body = """\
          <div class="card">
            <div class="card-content">
              <div class="media">
                <div class="media-content">
                  <p class="title is-4"><a href="https://blog.maxbridgland.com/blog/{direct_url}">{title}</a></p>
                  <p class="subtitle is-6">{upload_date} by {name}</p>
              <div class="content">
                <center><a href="https://blog.maxbridgland.com/blog/{direct_url}">Show More</a></center>
    all_keys = db.keys()
    keys = []
    for key in all_keys:
        if len(key) < 5:
    for key in keys:
        obj = json.loads(bytes(db.get(key)).decode('utf-8'))
        content = obj['content']
        title = obj['title']
        upload_date = obj['upload_date']
        new_key = str(key).replace('b', '').replace("'", '')
        obj['home_html'] = home_post_body.format(title=title, upload_date=upload_date, content=markdown(content[:120] + "......", extras=["fenced-code-blocks"]), name=app.config['name'], direct_url=new_key)
        db.set(key, json.dumps(obj))

Let's go through and analyze what this function is doing. First I have a formatted HTML string that I will use to generate the card on my site. This HTML is updated compared to what we used to have. After that I pull all my keys from the db and grab all blog keys. I use len(key) < 5 to check that the key is not a long string like the keys I don't want in my db (one of the flaws I've found with Redis is that you can't have "collections" like in MongoDB). I iterate over those keys and append them to a new list with just the blog keys. Then I iterate over that and grab the value from the db. They are all json strings so i load those string with the json module into a Python object. I then modify this python object and reset the key in the database. Ez Pz Migration :)

Creating a Basic REST API with Falcon and MongoDB

Published on 09/19/19 at 10:36 EST by Max Bridgland


Recently I started working with the Falcon Framework to make an API for a URL shortening site. A friend of mine showed me the framework and I decided to give it a try. It's a bit of a learning curve from other common frameworks like Django or Flask, but once you get the hang of it, it's super easy to work with.

In this article I'll go over the basic setup of a Falcon REST API with MongoDB support. This is the first part to a multipart series on how to build a complete Dart, Falcon, MongoDB webapp. To start, we will want a file structure like so:

- app/
|- controllers/
|- resources/
|- models/
|- __init__.py
|- settings.py
- app.py

All of our logic will live inside of the app/ folder. Inside this app folder we have a controllers/ folder which will hold the helper functions and logic for communicating with our database and performing internal functions, theresources/ folder which will hold all of our routes, and themodels/ folder will hold the logic for setting up our models in our database. The settings.py file contains our configuration for the application where we will store our variables for credentials and what not. Make sure you include an init.py file in every folder insideapp/. The top level init.py file will include our base setup and definition of our application. Let's start with getting the dependencies needed to run our application. Make sure you have mongoDB installed on your system, or have a database hosted on a server. If you have it hosted on your server then the HOST constant in our settings.py folder will be the IP of that address. If you have the database hosted locally for development purposes you will use localhost as the HOST constant.

You can check if you have mongoDB installed by running:

mongod --version
If you see something like:
root@mainbox:~# mongod --version
db version v3.2.22

then you should be set. There will probably be more information about your environment but the first line is all that matters. For this I'll be using mongoDB version 3.2.22.

To get setup with our python dependencies we will start by creating a virtualenv:

pip3 install virtualenv
virtualenv .venv
source .venv/bin/activate # or .venv/Scripts/activate on Windows
pip install falcon falcon-jsonify mongoengine

If you are on Unix or Linux you will also need to install a webserver to run the application. You can use uwsgi or gunicorn. If you are on Windows you will want to use the waitress module.

To install uwsgi or gunicorn:

pip install uwsgi
pip install gunicorn
To install waitress and hupper for hot reloads:
pip install waitress hupper

Waitress doesn't include a hot-reload option by default like uwsgi or gunicorn so you must use hupper in order to have this feature (it makes development a lot easier but it is not necessary).

Now that we have all of our dependencies installed we can start configuring our application. Open up our settings.py file inside app/. Set it up like the following snippet replacing your information:

import os
import flask_jsonfiy
dbcfg = {
    'host': 'localhost', # or external server address
    'port': 27017,
    'username': os.environ.get('MONGO_USER'),
    'password': os.environ.get('MONGO_PASS'),
middleware = [

This is all we need for now. If you end up using middleware or need to configure other options do it inside this file and import it from your other modules.

Now we can setup our core init.py file inside the app/ folder. Set it up like so:

import falcon
import mongoengine as mongo
from app.settings import middleware
app = falcon.API(middleware=middleware)
db = mongo.connect(
    'development', # This will be the name of your database

For now this all we need until we add an endpoint in resources/. Now we can setup our basic User model for the website. This is the only model we will be working in this part of the tutorial. To start create a file inside models/ called notes_model.py. You should setup your file like so:

from mongoengine import *
import datetime
class NoteModel(Document):
   title = StringField(max_length=200, required=True)
   body = StringField(max_length=32, required=True, unique=True)

There isn't much to the model for now but you'll add to this later. Now we can move onto setting up our basic register endpoint. Open up the resources/ folder and add a file named notes.py. Inside of this file we will add the logic to setup our handlers and responses. You should also make a file named notes.py inside the controllers/ folder. These files will work together. Setup the resources/notes.py file up like so:

import falcon
from app.controllers.notes import NotesController
from app import db
from app.models.notes_model import NoteModel
class GetNotes(object):
    def on_get(self, req, resp, id):
        resp.status = falcon.HTTP_200
        notes = []
        notes_obj = NoteModel.objects(title=title)
        for object in notes_obj:
        resp.json = {
            'notes': notes
class UploadNotes(object):
    def on_post(self, req, resp):
        title = resp.get_json('title', dtype=str)
        body = resp.get_json('body', dtype=str)
        notes_obj = NoteModel(
            title=title, body=body
        resp.status = falcon.HTTP_201
        resp.json = {
            'message': 'Your Note Has Been Posted!',
            'status': 200,
            'successful': True

Now that we have our resource setup we can open our core init.py file in the app/ folder. You will want to add these lines to the bottom:

from app.resources.notes import *
app.add_route('/notes/', UploadNotes)
app.add_route('/notes/{}', GetNotes)

Unlike other frameworks you aren't going to be returning anything. This is all we need for a basic REST API. We can now communicate with our mongoDB database by sending requests to our server.

Now we can start our webapp:

With Gunicorn:
gunicorn -p 8000 --reload app:app
With uWSGI:
uwsgi --http :9090 --wsgi-file app.py
With Waitress and Hupper:
hupper -m waitress --port=8000 app:app

Your server should be running on http://localhost:8000/ now. You should try to post something to the /notes/ endpoint which will allow you to add a note.

Sending Text Messages From An E-Mail With Python

Published on 09/19/19 at 01:14 EST by Max Bridgland

This is the method I use for my 2FA to not have to use Twilio or anything else. This is free except for the fact I'm using an enterprise domain email hosted on G-Mail.

Here is my send_sms function:

import smtplib
from email.message import EmailMessage
from app import app

mail_settings = {
    "MAIL_SERVER": 'smtp.gmail.com',
    "MAIL_PORT": 465,
    "MAIL_USE_TLS": False,
    "MAIL_USE_SSL": True,
    "MAIL_USERNAME": '[email protected]',
    "MAIL_PASSWORD": 'password-here',
    "MAIL_END": '@tmomail.net' # TMobile Email

def send(message):
        # Replace the number with your own, or consider using an argument\dict for multiple people.
    if app.testing:
        to_number = '555-555-0000{}'.format(mail_settings['MAIL_END'])
    else: # pragma: no cover
        to_number = 'XXX-XXX-XXXX{}'.format(mail_settings['MAIL_END'])
    auth = (mail_settings['MAIL_USERNAME'], mail_settings['MAIL_PASSWORD'])

    # Establish a secure session with gmail's outgoing SMTP server using your gmail account
    server = smtplib.SMTP( "smtp.gmail.com", 587 )
    server.login(auth[0], auth[1])
    msg = EmailMessage()
    msg['Subject'] = " "
    msg['From'] = "Me!"
    msg['To'] = to_number
    # Send text message through SMS gateway of destination number