Automation on Mohseen

Automate Publishing Blogs with GPT-4

Tue, 03 Oct 2023 00:00:00 +0000

Summer 2023

It’s a long summer, and AI has been a buzz ever since Chat GPT was introduced to the general public. Since then it’s been a roller coaster ride of AI advancements, and the latest one is GPT-4. While there is much heated debate on whether it’s out to replace us or turn into a skynet version of virtual overlords, I wanted to see if it’s any good at writing tech blogs.

Enabling Airflow API

Tue, 30 Nov 2021 14:30:23 -0800

Introduction

Hey folks, previously I wrote a blog post about how to setup Airflow and how it can streamline data pipelines and a great replacement for cron tasks.

In this post I’ll go over how to enable their api to trigger dags through nodes vs manually or scheduled runs.

By default after installing Airflow, the api is disabled for all requests. From Airflow docs, a few apis are experimental and need additional security like basic auth or only accept requests from trusted ips.

Airflow Cron on Steroids

Wed, 17 Nov 2021 11:25:55 -0800

Airflow

Airflow is a popular open-source framework for running data pipelines, but also really great at running tasks/scripts on a schedule. Cron has supported scheduling tasks for a while, but it’s really hard to configure and monitor it correctly. Fortunately for me, Airflow was a lot easier to setup DAGs (Directed Acyclic Graphs) and dependencies needed between tasks.

Overview

Airflow has a few key features that make it easy to use: