Michael Ruhl's Picture

Michael Ruhl

9 posts

Merging IOptions<T> settings from multiple sources

.NET Core rocks for so many reasons. One thing I've been using recently is demoing updates and real-world debugging via Azure and then deploying to my VPS in a Docker container somewhere else. I don't generate enough traffic to make an Azure web service more affordable than a VPS, which is why I don't run everything from the cloud. So, QA is a WebApp running on Azure in a cheap tier, which I use my credits from my MSDN subscription for, then my production is in Docker running from the same box as this blog.

.NET Core makes configuration between these two sources so simple. If you know how to use environment variables in docker, in an OS, and in a cloud service (like Azure), then you can deploy any of those ways without changing anything at build or run time (aside from initial set-up).

Here's the options class I'm dealing with:

public class EmailSettings
{
    public string MailGunApiKey { get; set; }
    public string ApiBaseUri { get; set; }
    public string From { get; set; }
    public string Domain { get; set; }
    public string RecaptchaApiKey { get; set; }
    
    public bool IsValid()
    {
        return !string.IsNullOrEmpty(MailGunApiKey) &&
            !string.IsNullOrEmpty(ApiBaseUri) &&
            !string.IsNullOrEmpty(From) &&
            !string.IsNullOrEmpty(Domain) &&
            !string.IsNullOrEmpty(RecaptchaApiKey);
    }
}

Super simple, I have a form on a web page to email me. I implemented ReCaptcha v2 to cut down on spam and I use the MailGun API to send the email. ApiBaseUri is the base API URL for MailGun, From is the email I'm sending to (me), Domain is the domain I'm sending from, and the two ApiKey properties are for each API, respectively.

The hurdle I had to jump across yesterday dealt with IOptions<T>. Typically, you can easily deserialize an object from json into your settings object with a one-liner. Then you can use Dependency Injection to access it in a controller. Here's an example:

//Startup.cs

public void ConfigureServices(IServiceCollection services)
    {
        services.Configure<EmailSettings>(Configuration.GetSection("EmailSettings"));
    }

This works great when all of your properties in that Options class are in one location with equal security requirements. As you can see, I created a class, though, where some properties could be stored in a file and others (like the private api keys) needed to be stored as environment variables, to avoid being committed to source. What I wanted was the portability and ease to store many config variables in a file with the security of keeping some out of source and directly on the host.

So, I came up with a solution of blending sources into one class so that they would be available as one. In order to load my api keys separately, this is what I added in place of the statement noted above:

public void ConfigureServices(IServiceCollection services)
{
//...
    services.Configure<EmailSettings>(options=> {

        options.ApiBaseUri = Configuration["EmailSettings:ApiBaseUri"];
        options.From = Configuration["EmailSettings:From"];
        options.Domain = Configuration["EmailSettings:Domain"];
        options.MailGunApiKey = Configuration["EmailSettings:MailGunApiKey"];
        options.RecaptchaApiKey = Configuration["EmailSettings:RecaptchaApiKey"];
        if (string.IsNullOrEmpty(options.MailGunApiKey) &&  Configuration.GetValue<string>("MAILGUN_API_KEY") != null)
        {
            options.MailGunApiKey = Configuration.GetValue<string>("MAILGUN_API_KEY");
        }
        if (string.IsNullOrEmpty(options.RecaptchaApiKey) && Configuration.GetValue<string>("RECAPTCHA_API_KEY") != null)
        {
            options.RecaptchaApiKey = Configuration.GetValue<string>("RECAPTCHA_API_KEY");
        }

    });
    //...
}

From this I'm getting exactly what I want from each "source". If I want to set variables in my file, I can and they will populate and not be overwritten by environment variables. Otherwise, it checks for environment variables too. I listed out all of these properties to show that you can also mix and match what you grab from JSON on a per-property level.

To finish this up, I call the EmailSetting.IsValid() method in my page generation to decide whether to build the html for the email form if one of these things is missing. That way, I won't have unnecessary broken calls to my API and it's also a quick way to see if something is missing. Combining this method in production with UserSecrets in Development is an excellent way to manage your dev and prod configs separately and securely.

More info on Configuration in .NET Core 2 can be found here.

Sql Server and Oracle: A Love Story Pt 1

I was given the most honorable task of migrating a fairly large SQL Server database over to Oracle 12c at my day job. In doing so, I have gleened a ton of knowledge about why you should never do this. It's a tough line: future proofing your app versus using proprietary tech for one platform that may not transfer over to another platform. We have a good mix of the latter.

I have found one particular great thing out today. Oracle's Sql Developer has a fairly rudimentary migration feature for moving Sql Server and Sybase databases over to their platform. The trick to grab the ddl straight from the source database is to import it. This can be done by downloading a jdbc driver and telling Sql Developer about it link.

What I ran into today was having an instance on a server. In order to validate scripts and get things moving, I set up an identical Sql Server on my dev machine with the same schema as the one I'm trying to migrate. As I found with an instance, there was no where to put it in the box:
newdb

Hostname is JUST for the server name. If you have an instance name, you're out of luck. So I started thinking about the connection string this form is no doubt generating with these boxes. Eureka! You can inject the instance into the connection string by appending it to the port. So in your port, just add 1433;instance=sqlserver. That appends the connection string and you can now connnect!

Once the Sql Server connection is made, you can right click on it and select 'Migrate Database.' From there, you'll learn as I did that that tool is nothing short of a pain in the rear.

Blog Theme Update

I've moved from the default theme. I was using it even with the old site. I found a really sleek theme called StayPuft. It just looks like the author hasn't updated it for Ghost v1 yet. I forked the repo, which is available here. If you're having trouble with StayPuft, perhaps give my fork a try. Keep in mind it has a different color scheme.

There were a few technical issues I had to overcome with this theme. They were quick to resolve and I had a good time doing it.

max_body_size in nginx.conf

I hadn't set this. I was trying to upload this theme but the size is too big (2mb). So, I had to update nginx.conf's http section and include client_max_body_size 2m;. The default is 1mb. This file is located in /etc/nginx. I am running nginx in a docker container so I just had to edit the file in my docker volume and restart nginx. It was super-simple.

obsolete helpers in Ghost

The main reason I had to fork the theme was because it was written for v.11. In v1 the ghost team changed some helpers involving images. The absolutely awesome part about it is that when you upload a theme, ghost tells you all the parsing errors in a great format that's easy to debug and fix. I am so very impressed with ghost's features.

In closing, I'll be tweaking this theme as I go. I don't like where the search results are for ghostHunter. I want to add some features and adjust the colors as well. The repository will stay current!

Welcome to blog v2!

Welcome to blog v2!

I committed the cardinal sin of devops: upgrading without backing up. It was quite unfortunate. Sometimes I think, "it's just my site, who cares," but I lost a few good hours of explaining something really cool.

To be fair, this is the first time it's happened for real. In prior iterations, I've backed up my database and restored it. This time I figured since the database was persisted on the filesystem, it would just start back up ok. Upon closer inspection, it appears my ghost installation, at some point, reverted to a vanilla install and started using the sqlite local database again. I'm not exactly sure how, but it started back up from scratch, so that .db file was brand new when I finally inspected it. Half my posts were in that while half were in a mariadb database. I have the mariadb posts. I don't have the sqlite posts. Those, I suspect, are gone. So, backup and backup some more. Thinking "well, my data is persisted" didn't cut it. An updated image started up and cut that theory down pretty quickly.

The good news! Yes, there's good news! I have updated containers running. I sincerely hope I've learned my lesson and will read release notes before docker pulling any new containers. I'll re-post my configuration with updated settings. It's worth reviewing. A few things changed since updating. For the record, I've updated all containers along with the docker engine. So pretty neat. And I've added a few perks. This new version of Ghost is pretty snazzy so I don't regret much.

I might have to invest in a dev environment, even for this dinky site. Making changes directly in production is never a good idea without testing it first. Docker makes it easy, too. I should post a set up of that. More to come...