MySql backup and restore

Prerequisites include a MySql server, probably Linux, shell access and MySql services installed and running.

 Backing up and restoring large MySql databases is more difficult than working with small databases. Small databases are easily dealt with using the ubiquitous phpMySqladm administration tool associated with most administrator situations but when the database exceeds a certain size between 2 and 95 Megabytes, using html to download and upload the backup files becomes less desirable than an SSL transport such as sFtp.

1. Determine your MySql database name, MySql user name and password, your domain for mysql (sometimes localhost suffices)

mysqldump -uUserName -p -h mysql.MyDomainName.com DatabaseName >file.sql

This command ends with >file.sql, which redirects the output to the file named file.sql in the current working directory.

2. Use the mysql command to restore the backup file. In this case, DatabaseName could be a different database within the same server, allowing you to make test database, or it could be the same database name on another MySql server.

mysql DatabaseName -uUserName -p <file.sql

Since some versions of html make have difficulty with the ‘less than’ sign, here is another way of doing the same thing.

cat file.sql | mysql DatabaseName -uUserName -p <file.sql

In both these cases, the command is prompting for the password with the -p option.

Boto Python Amazon EC2

Amazon web services API for python, Boto 2.6.0, 2.9.5 and 2.9.6 for Python 2.7

 First, a sample boto python 2.7 script tested on Dreamhost with their installed boto 2.6 as well as boto 2.9.5 and 2.9.6 on Heroku. The Amazon keys are imported from our custom made cred2 module, which you will make yourself for your keys.

#!/usr/bin/python

import boto.ec2
import cred2

conn = boto.ec2.connect_to_region("us-east-1", \
    aws_access_key_id=cred2.awsaki(), aws_secret_access_key=cred2.awssak() )

# conn.start_instances('i-59999x2x')

reservations = conn.get_all_instances()
print reservations
for res in reservations:
  for isx in res.instances:
    print isx

 

substitute what the python 2.7 functions return with your keys

Your amazon keys can be found in your Amazon Web Services account

#!/usr/bin/python

def awsaki():
  return 'AKXXXXXXXXXXXXXXXXXX'

def awssak():
  return 'vvv999ccc888xxx777zzz666lll555kkk444z/'

We used awsaki which means Amazon Web Services Access Key ID or A.W.S.A.K.I. which is the best terminology to use in search engines to find information on digging this out of your account

Google this

Amazon Web Services Access Key ID

We used awssak which means Amazon Web Services Secret Access Key or A.W.S.S.A.K. which is good terminology to use in search engines to find information on digging this out of your account

Google this

Amazon Web Services Secret Access Key

 

Python 2.7 script to start an AWS instance that is stopped

#!/usr/bin/python

import boto.ec2
import cred2

conn = boto.ec2.connect_to_region("us-east-1", \
    aws_access_key_id=cred2.awsaki(), aws_secret_access_key=cred2.awssak() )

reservations = conn.get_all_instances()
rsx = reservations[0]
hsx = rsx.instances[0].start()

This works great if you have an AWS account with one instance, which is the one you want to start. So we aren’t creating an instance here, we are just starting an instance that has been stopped.

 

Now a minimal example in flask to pass boto results to the jinja template rendering modules

@app.route('/aws')
def awshm():
 x = zzcode.botoclass(); cvx = x.botofunc()
 return render_template('aws.html',navigation=navfunc(), contentVar=cvx )

 

Here we use boto in the class

class botoclass:
 def botofunc(self):
  import boto.ec2
  import credec2
  conn = boto.ec2.connect_to_region("us-east-1", \
    aws_access_key_id=credec2.awsaki(), aws_secret_access_key=credec2.awssak())
  reservations = conn.get_all_instances()
  cvx = list()
  for rsx in reservations:
   for isx in rsx.instances:
    cvx.append({'reservation': str(isx), 'status': str(isx.state), \
              'dns': str(isx.public_dns_name), 'ip': str(isx.ip_address) })
  return cvx

 

In the midst of a template in the templates directory named aws.html is this table

<TABLE>
<TR><TH>Reservation </TH><TH>Status </TH><TH>Domain </TH><TH>ip </TH></TR>
{% for item in contentVar %}
<TR>
 <TD>{{ item.reservation }}</TD>
 <TD>{{ item.status    }}</TD>
 <TD>{{ item.dns      }}</TD>
 <TD>{{ item.ip       }}</TD>
</TR>
{% endfor %}
              </TABLE>

 

Drupal, Joomla and WordPress

blogging and more powerful user definable website software, php and apache based cms

 

Drupal addin features, links at the end of each blurb will take you to the drupal project page

FileField helps with CMS applications requiring a great deal of files and robust file management. FileField also has a file manager API. filefield

WYSIWYG – improved ‘What you see is what you get’ editor

Drupal allows text and html editing, probably with the tinyMCE library, but WYSIWYG allows you to integrate your preferred editor, e.g. TinyMCE, YUI, CKEditor, FCKeditor, jWysiwyg, markItUp, NicEdit, openWYSIWYG, Whizzywig, or WYMeditor. wysiwyg

Webform – contact form, survey, poll, etc.

Webform allows you to quickly create a form, and can save the submission in a database table. webform

Google Analytics

If you have not installed Google Analytics on a website, you have been missing out on a robust and simple tracking tool that shows charts, emails reports, even analyses keywords used to find nodes on the drupal site. A simple plugin would only need the UA code for basic functionality. google_analytics

Ubercart – shopping cart, ecommerce

For Drupal, Ubercart was developed for usability, and can handle shippable real goods, music downloads, software downloads, memberships or subscriptions. The extension purports to offer integrated order processing including payment and shipping processes. wow! ubercart

Drupal is a CMS, or Content Management System, is open source, runs on LAMP and offers newbies the ability to quickly post their content on the internet without the constraint and boredom of wordpress.

Sprox form with Genshi

Make sure we have these;

pip install sprox
pip install genshi
pip install tw.forms

And we will be able to write a script to produce a form without getting too long.

from sqlalchemy import Column, Integer, String, create_engine, Date, Float, Text
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.declarative import declarative_base
from sprox.formbase import AddRecordForm, FormBase
from sprox.fillerbase import TableFiller
from sprox.tablebase import TableBase

# coding: utf-8

Base = declarative_base()
metadata = Base.metadata


class Customer(Base):
    __tablename__ = 'customer'

    id = Column(Integer, primary_key=True)
    customer = Column(String(6), nullable=False, unique=True)
    name = Column(String(40), nullable=False)
    zip = Column(String(255), nullable=False)
    phone1 = Column(String(31), nullable=False)

class newCustomerForm(FormBase):
    __model__ = Customer

class DBSession1():
    def __init__(self):
        dbhost = "x"
        dbuser = "x"
        dbpass = "x"
        dbdbas = "x"
        self.engine1= create_engine('mysql://' + dbuser + ":" + dbpass + "@" + dbhost + "/" + dbdbas)
        self.Base1 = declarative_base()
        self.Session1 = sessionmaker(bind=self.engine1)
        return None


if __name__ == "__main__":
    w = DBSession1()
    x = newCustomerForm(w.Session1)
    print x()

This should produce html to enter fields that can be the centerpiece of a template or fetched with javascript and placed in a div. We still have not populated the form with data for updating, but this will do for a create. We also have explicitly defined the model, and we really want to do that on the fly.

What is Genshi and why did it pop up?

This might actually be a weakness of the Sprox packages, but Sprox really likes Genshi to make that final push from a Python data structure to HTML code. I call it a weakness, but this final transition is the realm of a template architecture. Popular Python templating specifications include Jinja2, Chameleon(Zope), Mako, etc. The weakness here is that it really likes Genshi, and it is not obvious how to use, for example, Mako instead.

Python SQLAlchemy MySql Columns

Lets switch to SqlAlchemy to get columns

#!/bin/python27

from sqlalchemy import create_engine, Table
from sqlalchemy.orm import scoped_session,sessionmaker
from sqlalchemy.ext.declarative import declarative_base
import sqlite3

class DBSession1():
    def __init__(self):
        dbhost = "x"
        dbuser = "x"
        dbpass = "x"
        dbdbas = "x"
        self.engine1= create_engine('mysql://' + dbuser + ":" + dbpass + "@" + dbhost + "/" + dbdbas)
        self.Base1 = declarative_base()
        return None

if __name__=="__main__":

    x = DBSession1()
    customers = Table('customer', x.Base1.metadata, autoload=True, autoload_with=x.engine1)
    print customers.columns

Once again, we get a list of columns.You will need to put your MySql database credential in for this to work, often times the host is localhost.

Here is a loop to look at even more information on the database columns. You can tack this onto the end of the code segment above.

    for col in customers.columns:
        print col.name,col.type,col.nullable,col.primary_key,col.foreign_keys

ubuntu postgres

Python 2.6, Python 2.7, an Ubuntu
kernel can be prepared in EC2 for
Django and Postgresql needs some major libraries not
often included in the core set of libraries. Gcc is the Gnu compiler.
Recent Python versions on instances are Python 2.6.4 and Python 2.7.2

Use apt-get to install postgres, gcc and these libraries

sudo apt-get update
sudo apt-get install postgresql
sudo apt-get install gcc
sudo apt-get install python-dev
sudo apt-get install libpq-dev
sudo apt-get install python-virtualenv
sudo apt-get install python-psycopg2

You probably want a virtual environment

source bin/activate

or

virtualenv nameofnewthing
source nameofnewthing/bin/activate

and pip install Django, gunicorn and dj_database_url

pip install Django
pip install gunicorn
pip install dj-database-url

Flask Python Heroku

 lets start building source code

cd source # MyHerokuProject/source 

We need to create two files for heroku to spin up our application;

pip freeze > requirements.txt
cat >Procfile
web: python hello.py

create your index.html equivalent

cat >hello.py
import os
from flask import Flask
# import BeautifulSoup
import urllib2
import re

app = Flask(__name__)

@app.route('/')
def hello():
return 'Hello World!'

@app.route('/gb')
def goodbye():
return 'Good Bye'

if __name__ == '__main__':
# Bind to PORT if defined, otherwise default to 5000.
port = int(os.environ.get('PORT', 5000))
app.run(host='0.0.0.0', port=port)

and lets git it to heroku

git init 
git add .
git commit -m 'init'
heroku create
git push heroku master 

Heroku Setup for Python and Flask

Getting virtualenv – Setuptools, pip, easyinstall on Ububtu

  • Install python setuptools on Ubuntu
  • Install pip on Ubuntu
  • easy_install on Ubuntu
  • virtualenv on Ubuntu – yes, get used to it.
sudo apt-get install python-setuptools
sudo apt-get install python-pip
  • Install Python development libraries, could be optional but we will use it here.
sudo apt-get install python-dev
  •  Get virtual env
sudo apt-get install python-virtualenv

You might also need a public key from your Ubuntu workstation. If your workstation is secure, you can just hit enter for the passphrase.

ssh-keygen -t rsa

Install the heroku toolbelt.

  • Install Heroku toolbelt on Ubuntu
  • Install Heroku toolbelt on AWS running Ubuntu
wget -qO- https://toolbelt.heroku.com/install-ubuntu.sh | sh

Now your 127.0.0.1 server is in sync with Heroku libraries.

Lets start a project.

mkdir MyHerokuPyProject
mkdir MyHerokuPyProject/source

Now, logout and login in to get heroku in the PATH, or do this;

 sudo su ubuntu

Log into heroku;

heroku login

You need to make heroku aware of the public key;

heroku keys:add

Ok, let’s make sure everything is set so far by starting up virtualenv

Lets use our project directory;

cd MyHerokuPyProject 

 We will add a virtual environment to our Python Heroku project;

virtualenv vedir --distribute
source vedir/bin/activate

Now we can get bottle like this. Notice that I am getting the bottle library, not the bottle.py library. It’s the same thing, but if you try to fetch bottle.py it will fail. Easy_install is a pythoin installation system, so it knows that that which we seek is a .py hootinanny.

pip install Flask

Add the Python image library, which requires the development library;

pip install pil

— or —

sudo apt-get install python-setuptools
sudo apt-get install python-pip
sudo apt-get install python-dev
sudo apt-get install python-virtualenv
ssh-keygen -t rsa
wget -qO- https://toolbelt.heroku.com/install-ubuntu.sh | sh
mkdir MyHerokuPyProject
mkdir MyHerokuPyProject/source

log out and log in


heroku login
heroku keys:add
cd MyHerokuPyProject
virtualenv vedir --distribute
source vedir/bin/activate
pip install Flask
pip install pil

Python Ubuntu Heroku edit

Assuming your project directory is “MyHerokuProject

cd MyHerokuProject

login

heroku login
source vedir/bin/activate
cd source

‘Git’ our project

heroku git:clone server-named-5678

Create new project provisioning a server on Heroku