Beginners guide to setting up a staging environment/ site on Linux, Apache, CentOS with password protection

So I need to setup a staging environment. I’m doing it on CentOS using the command line. This means I have a SSH connection to my server and have sudo privileges. But before that let’s begin by settings up a DNS entry in a control panel – if you don’t have that then you need to contact your web host.

This is what we want to create: a subdomain called “thestage” which will make the URL for access “thestage.yourlocation.com”. 300 is the TTL(time to live) and I’m not sure whether it’s right. Since it’s inaccessible to the public I’m going to experiment with that value for a little while.

thestage 300 IN CNAME yourlocation.com.

If it’s not an IP address then you need a full stop at the end.

I created a directory on this server for the staging contents. This will make it a top level directory like /var /etc /usr etc. It felt appropriate.

mkdir /thestage

I setup a vhosts (virtual hosts) conf file. This essentially means we can relay different addresses through the httpd (HTTP daemon) like Apache to different folders. This is how a lot of web hosting is conducted. So if you have access to vhosts then you can do this, if you don’t you’re probably on a hosting package that limits your ability to do things like this. Talk to your hosting provider and probably don’t continue reading the rest of this until you can do the aforementioned.

Because my httpd.conf (httpd configuration file) includes all ‘.conf’ (configuration) files in a directory of vhosts configuration files I can make a single instance specific/ bespoke version by creating a new virtual host configuration file (see below). If I didn’t/ couldn’t do this I would have to include it in the httpd.conf file which is possible but  can get messy if you have a lot of information and a lot of virtual hosts. The httpd.conf file is probably somewhere like /etc/httpd/conf/httpd.conf. (if you can’t find it do this: “cd /” … “locate httpd.conf”. Instead I put one here /etc/httpd/vhosts.d/thestage.conf where “thestage.conf” is the name of the file and vhosts.d is a directory being included because of the following line in httpd.conf:

Include vhosts.d/*.conf

I put this inside it to just test everything works first off:

<VirtualHost *:80>
    DocumentRoot /thestage
    ServerName thestage.yourlocation.com

    # Other directives here

</VirtualHost>

Since we’ve made a change to httpd we need to restart it

If your user has it setup – not guaranteed (sorry, look elsewhere for that) – then use this:

service httpd restart

Otherwise (or if it doesn’t work) use the file path too:

/etc/init.d/httpd restart

now go to thestage.yourlocation.com and you should see the default Apache page.

So go to /thestage and create index.html (this is the command line way, you could just use a program like WinSCP to create it and edit it in a GUI).

touch index.html

Then open that file to add contents

sudo vi index.html

(assuming you can and do use sudo, not discussed here)

Press [i] to enter “insert” module

Write anything, e.g.

Hello, world!

Press [Esc] to enter command mode

Type (they’re capitals btw):

ZZ

and press enter, this saves and quits

Refresh your browser and you should then see:

Hello, world!

So since that works we can get on with password protecting thestage.yourlocation.com. Since it’s a staging site it’s essentially still testing and we don’t want the world seeing it!

So the password advice in this Apache link didn’t work when I first tried it presumably because the path or something wasn’t set for me to use it so I had to do use the following command: locate htpasswd

# locate htpasswd
/etc/.htpasswd
/usr/.htpasswd
/usr/bin/htpasswd
/usr/share/man/man1/htpasswd.1.gz

The one I wanted is /usr/bin/htpasswd

So being in the directory with the program:

cd /usr/bin

we will almost be ready to use the command from the Apache link above. I have decided to not use /etc/.htpasswd so I will make a new file in my directory

 /etc/httpd

If you want to put it somewhere else which doesn’t yet exist then you will need to use mkdir with the -p argument to make the parent directories. For example:

# mkdir -p /usr/local/apache/passwd

But, still being in the /usr/bin directory with the htpasswd program (enter the command “pwd” if you’re not sure if you’re in that dir) type the following to create the new .htpasswd file and add a password to be used with yourusername (your user name) as below. If you’ve already got a .htpasswd file as in /etc/.htpasswd then you don’t need the -c argument.

htpasswd -c /usr/local/apache/passwd/.htpasswd yourusername 

Then you’ll need to enter and re-enter your password. This should work, if it doesn’t you might want to try another directory.

Then you’ll need to change the file ownership:

chown apache:apache /usr/local/apache/passwd/.htpasswd

Then create a .htaccess file in your your directory /thestage and add the following lines:

AuthType Basic
AuthUserFile /etc/httpd/.htpasswd
AuthName "Enter password"
Require valid-user

Or wherever your file is in place of “/etc/httpd/.htpasswd” Save it and restart apache (httpd) again as before.

Refresh your browser and now you should have a login form.

Login…

et voilà!

Now, depending on your needs you have the ability to setup a new database (a staging version of your live one) and a new codebase (perhaps with Git, I’ll come to that later, if I go down that road…)

Thanks :)

 

 

Recommended resources:

http://httpd.apache.org/docs/2.2/howto/auth.html

http://code.tutsplus.com/articles/setting-up-a-staging-environment–net-29738

http://billpatrianakos.me/blog/2012/11/20/do-yourself-a-favor-set-up-a-staging-server/

http://en.wikipedia.org/wiki/Chown

http://www.debiantutorials.com/password-protecting-a-directory-with-apache-and-htaccess/

 

Drupalgeddon :(

Man, choose the worst week to go on holiday why don’t you?!

So this has happened and it’s a pain in the bum and there’s not even any sure fire why of knowing whether or not your site/server has been compromised. Even after updating the problem is that as is described in this post on Drupal Stackexchange patching or updating Drupal core does not mean you have rid yourself of the problem. So I’ve updated to Drupal 7.32 and run Drupalgeddon but now there’s every chance that the best thing to do is rebuild the server. Man, what a pain in the arse!

Good clear guide to get started on understanding Drupal’s AJAX form functionality

This is a good article posted by Randy Fay on the form API changes from AHAH to AJAX forms in Drupal. For some reason I find the AJAX form functionality of Drupal incredibly confusing. This might be because I’ve never really had a chance to get my fingers into AJAX in any capacity, let alone for Drupal. So the underlying concepts I know, loosely and on a high level, but I’ve not had a chance to practise. So if you’re really struggling, maybe you should ditch Drupal at this point and get some AJAX stuff working.

 

General error: 1005 Can’t create table Drupal error when creating foreign keys

I kept getting “General error: 1005 Can’t create table …” when trying to add foreign key integrity constraints to Drupal through MySQL and db_query(). I had the schema defined through hook_schema() and I was putting the db_query() which had the foreign key constraint definitions in hook_install(), I tried them in hook_update_N() too which is all by the book and correct (either way is legit).

It turns out my school boy error was that the types did not match…

http://stackoverflow.com/questions/20532430/general-error-1005-when-creating-a-key-constraint-laravel

Unavoidably helpful modules and resources for Drupal module developers

Form Builder module

https://www.drupal.org/project/form_builder

  • install all the modules that come with Form Builder Example
  • then for a GUI visit: www.YOURSITE.com/form-builder-example
  • build the form and export the code then add it to a form building function as in drupal_get_form(…)

Schema module

https://www.drupal.org/project/schema

  • install
  • then visit: www.YOURSITE.com/admin/structure/schema/settings
  • or just click “configure” on the row with the schema module on the module listing page
  • click ‘inspect’
  • from there you can export a database table that you can make the traditional way in SQL or through PHPMyAdmin or similar
  • with that export you can

Web development testing process, methodology and checklist resources

This is a collection of resources on testing procedures, processes, methodologies, checklists or just related. In my experience, when you’re out of university one of the first things that ends up being dropped is the rigour of testing. Deadlines won’t wait and unlike when you’re studying, there’s no way you want to be up until two o’clock in the morning drinking energy drinks. Let me tell you now that dropping testing is a mistake.

Keep your testing procedures in place, keep your logging process in place. Take the time to keep your Gantt chart up-to-date and all the way along keep the methodology documented and tidy. If you do not do these things you will kick yourself later. Like sick leave or christmas parties it’s simply something that must be an overhead in business that you take into account. This is the job.

Resources: (please note some of these are rather old-skool development house style and therefore slower and heavier to assimilate, should you bother to read them at all)

Also, another for the pile – setup a staging server

 

Adding an update to a Drupal custom module during development

Introduction

This is a post on how to write an update for a custom module that you have been developing on Drupal 7. To do this there are some prerequisites. Firstly – you must have used a ‘.install’ file (MY_MODULE.install) in your custom development (and you must have added that MY_MODULE.install file to the list of files in your MY_MODULE.info file). You must have then installed your module by way of the typical /admin/modules route. If you’ve not done that then you need to – you’ll reap the rewards later as Views, Entities and a whole heap of other functionality will open up to you but you must use the code that declares your schema to Drupal, inside your MY_MODULE.install file.

Updating the existing schema definition

The first step in the update process proper is to add your new definition to the hook_schema() function

/**
 * Implements hook_schema().
 */
function MY_MODULE_schema() {

...

Hook update: the hook_update_N() function

Now you will need to add that same definition to the hook_update_N() function, but the manner in which you add it is different.

Firstly write the function declaration. This will be your module name replacing the word ‘hook’ and then your update identifier replacing the capital N. This to be honest still confuses me – I think the documentation on it is extremely confusing. So when the first number didn’t work (7001) I just used 7002 instead.

function MY_MODULE_update_7002(&$sandbox) {
    $schema['your_new_table'] = array(
        'description' => 'this is your new database table created through hook_update_N',
        'fields' => array(
            'id' => array(
                'description' => 'the ID of your_new_table',
                'type' => 'serial',
                'unsigned' => TRUE,
                'not null' => TRUE,
            ),
            'text' => array(
                'description' => 'some kind of text as an example',
                'type' => 'varchar',
                'length' => '128',
                'not null' => TRUE,
            ),
        ),
    );
    db_create_table('your_new_table', $schema['your_new_table']);
}

Don’t forget to clear your cache (using drush, “cd” into the Drupal folder then type “drush cache-clear”, or …/admin/config/development/performance)

If that still hasn’t worked you might need to increment your update number.

Finally, making it happen on your system

So now all we need to do is backup our database and then visit /update.php and follow the instructions!

Recommended resource: this is a really good guide to the process (scroll down to the appropriate step that you’re on) https://www.drupal.org/node/150215

Recommended resources

Database functions to use in your update: https://api.drupal.org/api/drupal/includes!database!database.inc/7

hook_update_N() information: https://www.drupal.org/node/150215

Drupal.org Community Documentation on Writing .install files (Drupal 7.x): https://www.drupal.org/node/876250

The connection was reset error message in web browser like Firefox when there has been a change to Drupal usually in moving sites around

This is the page you want to look at:

https://drupal.org/node/1777128

To be specific, this post:

I had a similar problem that I could NOT figure out for a few hours. My Google search brought me here more than once but none of the comments above helped me figure out what was going on. I finally stumbled upon this other post:

https://drupal.org/node/1597820

FINALLY a fix that worked for me. I decided to add the link to this post just in case someone else is trying to find a solution to the same problem I was having. (hopefully this isn’t off topic!)”

 

 

Just downloading Microsoft’s SkyDrive, I mean OneDrive

Before I had a chance to get used to the SkyDrive, it’s now been re-branded as the OneDrive. How often do Microsoft plan to change major brands? It’s disconcerting to see this, it feels like an admission of a lack of control. I like Microsoft, I have absolutely no problems with them. The near-monopoly they held on the personal computing industry feels like nothing compared to the near-monopoly Google has on the internet, and Facebook has on social networking (or rather, our personal lives as they’d like us to believe). I say that because I feel I have to turn to Microsoft as the provider of both an acceptable price and an acceptable privacy policy. All of that said, I have no idea of either the price or the privacy policy of Microsoft in this mobile era of computing. Microsoft have just completed their Nokia purchase however so I may be speaking too soon (not that this has been a slow process!)

Article in Time.com on measuring user behavior with metrics like clicks etc.

I’m asked a lot about metrics for measuring traffic, engagement etc. etc. Even though on a daily basis I have to deal with complex developing, people management, daily requests and tasks I still find grasping truth behind the behaviour of our end-users/ readers a slippery kipper…

So I must make some time to finish reading the following article from Time.com on what we think we know about measurement, click-throughs, clicks etc. compared to pausing, engagement and how it all fits together with social sharing.

Time article “What You Think You Know About the Web Is Wrong”

Using Pathauto functions in a block to create lists of Taxonomy terms and their associated nodes/ content with a custom content type where one node could have multiple terms

The problem

I came across a need to write a custom block in Drupal 7 which dealt with Taxonomy terms in a custom Taxonomy vocabulary. In this block I needed to list all nodes that were associated with that Taxonomy term, the problem was that one node could have multiple terms associated with it in the Views field. I started out using Views for this but found no way to prevent the Taxonomy terms from being listed multiple times per node so I decided to reverse the approach and list the terms in the Taxonomy which had the specific vocabulary.

For the full explanation of the problem see this post on Drupal Answers.

In working on this I went in circles for about an hour creating invalid URL aliases to match the clean URL pathauto alias that I had generated when the content was first created. However what I needed was to obviously match the path that was created in the first place. I realised that the way that I was going to do this was by returning back to the Pathauto module and checking through that api.

The solution

This is how I ended up getting it to work:

$vocabularyID = $row->taxonomy_term_data_vid;
      $termID = $row->tid;

        $path = drupal_get_path('module', 'example_sites');
        include_once($path . '/example_sites.api.php');
        $nidArray = fetch_node_ids_by_term_id($termID);
        $path = drupal_get_path('module', 'pathauto');
        include_once($path . '/pathauto.inc');

        foreach ($nidArray as $nid) {
            $node = node_load($nid);
            $string = $node->title;
            $cleanTitle = pathauto_cleanstring($string);
            $url = 'category/' . $cleanTitle;
            //below is just applying a class for CSS styling etc.
            $options['attributes'] = array(
                'class' => 'category-with-term',
            );
            $link = l($node->title, $url, $options);
            print $link;
        }

Please be aware that fetch_node_ids_by_term_id($termID), here:

        $path = drupal_get_path('module', 'example_sites');
        include_once($path . '/example_sites.api.php');
        $nidArray = fetch_node_ids_by_term_id($termID);

…is actually just me consulting the database, you can replace all that with this…

    $nidArray = db_select('taxonomy_index', 'ti')
        ->fields('ti', array('nid'))
        ->condition('tid', $tid, '=')
        ->execute()->fetchCol();