I created a small helper script that encapsulates API calls for simple snapshot management. This can be used in cron jobs to trigger snapshot creation and cleanup.
Keeping manually created snapshots is the least backup security one should have on top of automatically created snapshots as the latter type is deleted together with the instance. In case of unintentional deletion of an RDS instance automated snapshots are of no help to restore the data.
In the first episode of my micro-controller adventures I’m blinking some LEDs by using a shift register. The goal of the exercise is to learn some basics about wiring the whole thing on the breadboard and write my first few lines of code.
And while at it I can also find out which tools I like to use.
Trying to wet my feet with micro-controllers I just started to play around with the ESP8266 which I find much more sexy than any Arduino that I’ve seen so far. Most probably because of the visual experience – I’m a visual guy when it comes to tech after all. ;-)
I’m also trying to start doing write-ups of my progress. I always wanted to do that but newer really got myself to actually do it. So here it is: My first post about my adventures in the world of micro-controllers.
The micro-controller I chose for prototyping is a MOD-WIFI-ESP8266-DEV from Olimex. First of all it is breadboard friendly. Secondly it exposes all of the chip’s pins and has 2MiB of flash memory which should be enough to house even more sophisticated projects. (Which I can’t judge yet because of lack of experience but I go with the rule of thumb that more usually is better.)
If you like to work with AWS on the CLI you can easily open the AWS by using a specially crafted link that logs you in using the credentials from your shell environment. To assemble the link you can use this little Python script:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
#!/usr/bin/env python import json import os import urllib import urllib2 data1 = { "sessionId" : os.environ['AWS_ACCESS_KEY_ID'], "sessionKey" : os.environ['AWS_SECRET_ACCESS_KEY'], "sessionToken" : os.environ['AWS_SECURITY_TOKEN'] } data2 = { "Session" : json.dumps(data1) } data3 = urllib.urlencode(data2) data4 = urllib2.urlopen("https://signin.aws.amazon.com/federation?Action=getSigninToken&" + data3) data5 = json.load(data4)['SigninToken'] print "https://signin.aws.amazon.com/federation?Action=login&Destination=https%3A%2F%2Fconsole.aws.amazon.com&SigninToken=" + data5 |
Wanna know whether your ELB has at least one instance in service or not?
Just add a CloudWatch Alarm to your ELB and have it send status changes to SNS (from where you can route it to whatever notification system you’re using).
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 |
{ "Parameters": { "AlarmSnsAccountId": { "Description": "Account ID of SNS topic to send Alarms to", "Type": "String" }, "AlarmSnsRegion": { "Description": "Region SNS topic to send Alarms to", "Type": "String" }, "AlarmSnsTopicName": { "Description": "Name of SNS topic to send Alarms to", "Type": "String" } }, "Resources": { "WebELB": { ... } "WebAutoScalingGroup": { ... }, "WebLaunchConfiguration": { ... }, "ELBHealthyHostCountAlarm": { "Properties": { "AlarmActions": [ { "Fn::Join": [ ":", [ "arn:aws:sns", { "Ref": "AlarmSnsRegion" }, { "Ref": "AlarmSnsAccountId" }, { "Ref": "AlarmSnsTopicName" } ] ] } ], "ComparisonOperator": "LessThanOrEqualToThreshold", "Dimensions": [ { "Name": "LoadBalancerName", "Value": { "Ref": "WebELB" } } ], "EvaluationPeriods": "3", "InsufficientDataActions": [ { "Fn::Join": [ ":", [ "arn:aws:sns", { "Ref": "AlarmSnsRegion" }, { "Ref": "AlarmSnsAccountId" }, { "Ref": "AlarmSnsTopicName" } ] ] } ], "MetricName": "HealthyHostCount", "Namespace": "AWS/ELB", "OKActions": [ { "Fn::Join": [ ":", [ "arn:aws:sns", { "Ref": "AlarmSnsRegion" }, { "Ref": "AlarmSnsAccountId" }, { "Ref": "AlarmSnsTopicName" } ] ] } ], "Period": "60", "Statistic": "Average", "Threshold": 0 }, "Type": "AWS::CloudWatch::Alarm" } } } |
Due to popular demand (read: colleagues asking) I put this little script online which I originally found when googling the web for a way to reduce the count of commits that contain syntax errors in my Puppet recipes.
It will not find all errors (e.g. cyclic or broken requires) but it reduces the risk of overlooking copy&paste errors or typos a lot.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
#!/bin/bash #Script to test puppet files have valid syntax. #Intended for use with hudson/jenkins. set -e set -u fail=0 #TODO: Run these in parallel - we have 4 cores. #TODO: Control the environment (through the config dir?). # We want to parse for all environments. # Is this being done, contrary to puppet report? #TODO: Even with --ignoreimport, some may be pulling in others, # meaning we're checking multiple times. all_files=`find . -name "*.pp" -o -name "*.erb"` num_files=`echo $all_files | wc -w` if [[ $num_files -eq "0" ]]; then echo "ERROR: no .pp or .erb files found" exit 1 fi echo "Checking $num_files *.pp and *.erb files for syntax errors." echo "Puppet version is: `puppet --version`" for x in $all_files; do set +e case $x in *.pp ) puppet parser validate --color=false $x ;; *.erb ) cat $x | erb -x -T - | ruby -c > /dev/null ;; esac rc=$? set -e if [[ $rc -ne 0 ]] ; then fail=1 echo "ERROR in $x (see above)" fi done if [[ $fail -ne 0 ]] ; then echo "FAIL: at least one file failed syntax check." else echo "SUCCESS: all .pp and *.erb files pass syntax check." fi exit $fail |
Update: I found the original source on GitHub
Have you ever tried working with a huge monolith of SVN repo that has lived for ages and seen a bazillion commits in its time? Then you might have experienced unpleasant times waiting for some process to chew through all that nasty pile of code.
Here is how I handle these behemoths – just checkout the subpath I need leaving all the history behind:
|
1 2 3 4 |
repopath=$1 folder=$2 lastrev=$(svn info $repopath | grep "Last Changed Rev:" | sed -e 's/.*: \(\d*\)/\1/') git svn clone -r $lastrev $repopath $folder |
This code snippet will fetch the latest revision the subpath was edited and do a sparse shallow checkout using git as client.
The following template will create a CloudFormation distribution with an AliasRecord pointing at it. It assumes your origin is already up and running. You don’t want to have the origin and CloudFront in the same stack anyways.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 |
{ "AWSTemplateFormatVersion": "2010-09-09", "Description": "CloudFront Stack with AliasRecord", "Outputs": { "Distribution": { "Value": { "Ref": "Distribution" } } }, "Parameters": { "DnsName": { "Description": "My DNS name", "Type": "String" }, "DnsZoneName": { "Description": "My DNS zone", "Type": "String" }, "OriginDnsName": { "Description": "DNS of the web origin", "Type": "String" } }, "Resources": { "Distribution": { "Properties": { "DistributionConfig": { "Aliases": [ { "Ref": "DnsName" } ], "DefaultCacheBehavior": { "ForwardedValues": { "QueryString": "true" }, "TargetOriginId": "Origin", "ViewerProtocolPolicy": "allow-all" }, "Enabled": "true", "Origins": [ { "CustomOriginConfig": { "OriginProtocolPolicy": "match-viewer" }, "DomainName": { "Ref": "OriginDnsName" }, "Id": "Origin" } ] } }, "Type": "AWS::CloudFront::Distribution" }, "DnsRecord": { "Properties": { "AliasTarget": { "DNSName": { "Fn::GetAtt": [ "Distribution", "DomainName" ] }, "HostedZoneId": "Z2FDTNDATAQYW2" }, "HostedZoneName": { "Ref": "ZoneName" }, "Name": { "Ref": "DnsName" }, "Type": "A" }, "Type": "AWS::Route53::RecordSet" } } } |
The best way to ensure some script/command running on shutdown is to use upstart which is the default init in CentOS6:
|
1 2 3 4 5 6 7 8 |
#/etc/init/instance-shutdown.conf start on starting rc RUNLEVEL=[016] task script # insert script action here end script |
In order to be able to properly diff and read your CloudFormation templates you want them to be in a harmonised shape:
This can be done on the shell: nice-cf.py < template.json > beautified.json
Or in vim using a keybinding:
|
1 |
map <C-j> :%!nice-cf.py<CR> |
This is the nice-cf.py Python script used to apply the transformations to the template:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
#!/usr/bin/env python import json import sys import re data = json.load(sys.stdin) jstr = json.dumps(data, indent=2, sort_keys=True) jstr = re.sub(r'{\s*("Fn::GetAtt")\s*:\s*\[\s*("\S+")\s*,\s*("\S+")\s*\]\s*}', r'{ \1: [ \2, \3 ] }', jstr) jstr = re.sub(r'{\s*("Fn::Select")\s*:\s*\[\s*("\d+")\s*,\s*({[^}]+})\s*]\s*}', r'{ \1: [ \2, \3 ] }', jstr) jstr = re.sub(r'{\s*("Fn::GetAZs")\s*:\s*("\S*")\s*}', r'{ \1: \2 }', jstr) jstr = re.sub(r'{\s*("Ref")\s*:\s*("\S+")\s*}', r'{ \1: \2 }', jstr) jstr = re.sub(r'{\s*("Name"):\s*("\S+"),\s*("Value"):\s*("\S*")\s*}', r'{ \1: \2, \3: \4 }', jstr) jstr = re.sub(r'{\s*("Key"):\s*("\S+"),\s*("Value"):\s*("\S*")\s*}', r'{ \1: \2, \3: \4 }', jstr) jstr = re.sub(r'{\s*("Key"):\s*("\S+"),\s*("Value"):\s*({[^}]*})\s*}', r'{ \1: \2, \3: \4 }', jstr) jstr = re.sub(r'\[\n\r?\s*([^\n]*)\n\r?\s*\](,?)', r'[ \1 ]\2', jstr) jstr = re.sub(r'\s+$', r'', jstr, flags=re.MULTILINE) jstr = re.sub(r'([^\n]*)\n\s*("\\n",)', r'\1\2', jstr, flags=re.MULTILINE) sys.stdout.write(jstr) |
By continuing to use the site, you agree to the use of cookies. more information
The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.
