Managing Tomcat with Chef (or at least trying to)

It turns out running Tomcat's catalina.sh start hides a bit of a gotcha: it doesn't always work with non-interactive shells. Specifically, it doesn't properly detach from the parent process and is killed when the remote session ends. For example, running knife-ssh "search query" "/path/to/tomcat/bin/catalina.sh start" will cause Tomcat to start and then be immediately killed when the SSH session ends. If you start Tomcat via a Chef recipe and you're executing chef-client remotely via knife-ssh, your results will vary depending on how you execute the command. Using Chef's execute resource seems to work okay, but running a System call inside a ruby_block resource does not. So running chef-client through an interactive SSH session will work, but launching chef-client via knife ssh will cause the Tomcat process to be killed with no output to the log files.

As a workaround, launch catalina.sh via nohup: nohup /path/to/tomcat/bin/catalina.sh start

Java XML-RPC Client with Apache XmlRpcClient

In this post I'll provide a working example of a Java XML-RPC client using the org.apache.xmlrpc.client package and explain how to parse responses.

First, Why XML-RPC?

If you're looking into creating an XML-RPC client, I sincerely hope it's to connect with a legacy system. For the most part XML-RPC APIs have fallen out of favor, replaced with JSON-based REST APIs and what not. But if, like me, you've been tasked with writing a client to consume a legacy API, you don't really have a choice.

Getting Started

Fortunately for Java developers (and most languages that aren't brand-spankin' new), XML-RPC support is pretty easy to come by. For this tutorial, I'll use the Apache library, which can be added to a Maven project with:

<dependency>
<groupId>org.apache.xmlrpc</groupId>
<artifactId>xmlrpc-client</artifactId>
<version>3.1.3</version>
</dependency>

Since what we're really writing is an integration test, I've also added in TestNG:

<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>6.8.13</version>
</dependency>

Now I can write a simple test class that starts like this: package com.techdagan.xmlrpcsandbox;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.util.Map;
import org.apache.xmlrpc.XmlRpcException;
import org.apache.xmlrpc.client.XmlRpcClient;
import org.apache.xmlrpc.client.XmlRpcClientConfigImpl;
import org.apache.xmlrpc.client.XmlRpcClientException;
import org.apache.xmlrpc.client.XmlRpcStreamTransport;
import org.apache.xmlrpc.client.XmlRpcTransport;
import org.apache.xmlrpc.client.XmlRpcTransportFactoryImpl;
import org.apache.xmlrpc.common.XmlRpcStreamRequestConfig;
import org.testng.Assert; import org.testng.annotations.Test;
import org.xml.sax.SAXException;

public class XmlRpcClientTest {

@Test public void textXmlRpcClientTest() throws Exception {

// Set up a client config that points no where (we're mocking anyway) XmlRpcClientConfigImpl clientConfig = new XmlRpcClientConfigImpl(); clientConfig.setServerURL(new URL("http://localhost/my/xml-rpc/endpoint"));

// Create our client XmlRpcClient client = new XmlRpcClient(); client.setConfig(clientConfig);

// Use our mock transport factories XmlRpcClientTest.MockXmlRpcTransportFactory transportFactory = new XmlRpcClientTest.MockXmlRpcTransportFactory(client); client.setTransportFactory(transportFactory); transportFactory.setResponse(new ByteArrayInputStream( ("<?xml version=\"1.0\"?>" + "<methodResponse>" + "<params>" + "<param>" + "<value><double>18.24668429131</double></value>" + "</param>" + "</params>" + "</methodResponse>").getBytes()));

Double answer = (Double) client.execute("add", new Object[]{ 1234, 56789}); Assert.assertEquals(answer, (Double)18.24668429131); }

Here's a breakdown of the code above:

  • Client classes are configured using object's implementing XmlRpcClientConfig, namely XmlRpcClientConfigImpl. You can set several properties of the config object, such as username and password (for basic auth, I believe), but you'll need to set the server URL at a minimum. To apply a configuration, simply pass a config object to the client's setConfig() method.
  • The client object will rely on three classes to do the actual work: a worker class, a transport factory, and a type factory. The worker class handles thread pooling, the transport factory generates the transport object (which does the actual server communication), and the type factory parses server's the response into an object. Aside from testing and extraordinary circumstances, the default implementations should be fine.
  • Since this is a test, we don't want to bother with setting up an actual XML-RPC server. Fortunately, the Apache XML-RPC package includes an abstract XmlRpcStreamTransport class that can easily be extended to implement a mock transport, which I've done.
  • Review the mocked response I've assigned. XML-RPC has a very strict format for requests and responses (checkout the Wikipedia article). Basically, each request names the method to call and provides an ordered list of parameters. Each response either define a fault or a method response. In the case of a method response, there is exactly one return value.
  • Finally, executing a request is fairly simple: pass a method name and array of objects to be used as parameters to the client's execute() method and cast the return value to the expected data type.

Return Values

The values returned by execute() have to be cast to the appropriate data type. There are eight, corresponding to the eight XML-RPC data types:

Stringstring

Integerint,i4

Doubledouble

Booleanboolean

Byte[]base64

Map<String,Object> (java.util.map)struct

Object[]array

Date (java.util.Date)dateTime.iso8601

(Okay, there's actually the possibility of 10 more, but you'll probably use these eight.)

In the case of struct and array, you'll need to do additional casting for each member, since there's no requirement other than a member have a name (for structs) and a value of one of the supported data types.

For me, the simple cast requirement was the missing piece. Having worked with more modern APIs and patterns, I was expecting to be able to implement a mapping interface and use generics to retrieve a domain-specific class. I guess I was over thinking it.

Wrapping Up

Perhaps surprisingly, that's really it for the client. Aside from executing the request, handling exceptions (thrown when communication fails or the server returns a fault response), and your own domain-specific mapping (you know, all that stuff), you're done.

Well, mostly, anyway. If you actually want the sample code above to execute, you'll need the mock transport factory and transport classes I created. So here are those:

private class MockXmlRpcTransportFactory extends XmlRpcTransportFactoryImpl {

protected InputStream response; protected XmlRpcClientTest.MockXmlRpcTransport transport;

protected MockXmlRpcTransportFactory(XmlRpcClient pClient) { super(pClient); this.transport = new XmlRpcClientTest.MockXmlRpcTransport(this.getClient()); }

@Override public XmlRpcTransport getTransport() { this.transport = new XmlRpcClientTest.MockXmlRpcTransport(this.getClient()); this.transport.setResponse(this.response); return this.transport; }

public void setResponse(InputStream stream) { this.response = stream; }

public ByteArrayOutputStream getLastRequest() { return this.transport.getLastRequest(); } }

private class MockXmlRpcTransport extends XmlRpcStreamTransport{

protected ByteArrayOutputStream request; protected InputStream response;

public MockXmlRpcTransport(XmlRpcClient client) { super(client); }

public MockXmlRpcTransport(XmlRpcClient client, InputStream response) { super(client); this.setResponse(response); }

public ByteArrayOutputStream getLastRequest() { return this.request; }

public InputStream getResponse() { return this.response; }

public void setResponse(InputStream response) { this.response = response; }

@Override protected void close() throws XmlRpcClientException { return; }

@Override protected boolean isResponseGzipCompressed(XmlRpcStreamRequestConfig pConfig) { return false; }

@Override protected InputStream getInputStream() throws XmlRpcException { return this.response; }

@Override protected void writeRequest(ReqWriter pWriter) throws XmlRpcException, IOException, SAXException {

this.request = new ByteArrayOutputStream(); pWriter.write(this.request); } }

PHP vs. The Developer: Encoding Character Sets

Ask most developers about character encoding and your apt to see one of three responses:

  1. a dramatic rolling of the eyes
  2. a looked of dazed incomprehension
  3. a pantomimed bullet to the brain

Ask a a PHP developer about character encodings and 99 out of 100 times you'll get either the second or third answer (the third being most likely if the developer has experience with them).

For a pretty decent description of the problems (and the best way to avoid them), check out: http://webmonkeyuk.wordpress.com/2011/04/23/how-to-avoid-character-encoding-problems-in-php/.

If avoiding the problem isn't an option, though, read on.

The Default PHP Encoding

PHP has two default encodings: the default character encoding for text read in from files and the default encoding for output (both to streams and files).

When text is read from a file, PHP retains the character encoding used in the file. So UTF-8 files will be read in as UTF-8 strings and ISO-8859-1 files will be read in as ISO-8859-1 encoded strings.

Note: PHP files are not excluded from the default-encoding rule. Specifically, strings defined in a PHP file will be encoded using whatever character set the file is stored in.

The other default encoding is that for output, which essentially means the value that is declared in the Content-Type header on any HTTP responses, and that one is set in the PHP INI file.

Getting Into Trouble

One of the most common problems with character encoding occurs when the default output character encoding is not consistent with the character encoding used in the server's PHP files. In that case, the server will be telling clients that response content is encoded in the wrong character set, which can result in scrambled text.

Another potential problem is inconsistent character encoding between PHP scripts and databases, which can also result in scrambled text.

Of course, the experienced PHP programmer would at least be aware of the multi-byte string extension designed to handle just these types of situations. Using mb_detect_encoding() and mb_convert_decoding, just such a developer may think she's got encoding licked. She'd be wrong.

Sadly, mb_detect_encoding() can flat out fail:

For UTF-16UTF-32UCS2 and UCS4, encoding detection will fail always.

Source: http://www.php.net/manual/en/function.mb-detect-order.php

It can also be flat our wrong. In fact, in my testing, mb_detect_encoding() always detected strings as either ASCII or UTF-8 (the difference being whether any non-ASCII characters were present).

Test Code

To test exactly how PHP and Apache handled character encoding, I set up a simple test script to encode/decode characters in HTTP requests and responses.

Here's the code:

<?php
$default_encoding = 'UTF-8';
$encoding = (array_key_exists('encoding', $_POST)) ? $_POST['encoding'] : $default_encoding;
$valid_encodings = array(
 'UTF-8', 'UTF-16', 'ISO-8859-1', 'ISO-8859-2', 'ISO-8859-3', 'ISO-8859-4',
 'ISO-8859-5', 'ISO-8859-6', 'ISO-8859-7', 'ISO-8859-8', 'ISO-8859-9', 'ISO-8859-10',
 'ISO-8859-13', 'ISO-8859-14', 'ISO-8859-15');
if (array_key_exists('string', $_POST)) {
 $string = $_POST['string'];
$encoded_string = mb_convert_encoding($string, $encoding, 'UTF-8');
$ch = curl_init($uri = 'http://' . 'portal.vidtel.lan' . $_SERVER['REQUEST_URI'] . '?test=1');
 curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
// Set the request data
 curl_setopt($ch, CURLOPT_POSTFIELDS, $encoded_string);
// Set the headers
 $headers = array(
 'Content-Type: text/plain;charset=' . strtolower($encoding),
 'Content-Length: ' . strlen($encoded_string));
 curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$response = curl_exec($ch);
} elseif (array_key_exists('test', $_GET)) {
 if (preg_match('/charset=(.*);?$/i', $_SERVER['CONTENT_TYPE'], $match)) {
 $match = strtoupper($match[1]);
$encoding = (in_array($match, $valid_encodings)) ? $match : $default_encoding;
 } else {
 $encoding = $default_encoding;
 }
$encoded_string = file_get_contents('php://input');
 $string = ($encoding == $default_encoding) ? $encoded_string : mb_convert_encoding($encoded_string, $default_encoding, $encoding);
 printf('"%s" was detected as being encoded with %s.' . PHP_EOL, $string, mb_detect_encoding($encoded_string));
 echo '"' . $string . '" was actually encoded using ' . $encoding . ': ' . $encoded_string;
 exit();
}
?><!DOCTYPE html>
<html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 <title></title>
 </head>
 <body>
 <h1>Provide a String and Character Set</h1>
 <form method="post">
 <p>
 <input name="string" <?php if (isset($string)) printf('value="%s" ', $string); ?>>
 <select name="encoding">
<?php foreach ($valid_encodings as $charset) : ?>
 <option<?php if ($charset == $encoding) echo ' selected="selected"'; ?>><?php echo $charset; ?></option>
<?php endforeach; ?>
 </select>
 <input type="submit" value="Test Encoding">
 </p>
 </form>
<?php if (isset($response)) : ?>
 <h1>Test Result</h1>
 <pre><?php echo $response; ?></pre>
<?php endif; ?>
 </body>
</html>

Implementing DCI in PHP

Disclaimer: This post is intended only as a means of clarifying my own ideas on implementing DCI in PHP. I haven't yet done any real research on if/how others in the PHP community are accomplishing this, but as a matter of practice, I prefer to get  my own ideas down before influencing them with the ideas of others. Once I have a better understanding of how best to implement DCI in PHP, I'll publish a new post and update this one with a link.

Introduction

DCI is one today's coolest cutting-edge programming models. It's so new that a lot of development communities are still trying to figure out how best to implement DCI in their language (the biggest challenge being run-time method injection, which happens to be a staple of DCI). PHP is no exception, and I've decided to tackle implementing DCI in PHP as both a thought experiment and, perhaps, my new way of doing things.

The DCI paradigm, as I understand it today, has three basic rules:

  1. Data (what the system is) should be encapsulated in a manner that only includes the  most basic functionality necessary to access the data (e.g. getters and setters).
  2. Context (or use case) expand data objects to include the methods/actions necessary to the specific use case through method injection.
  3. Interaction (what the system does) introduces that data objects to the context(s) necessary to accomplish a particular task or set of tasks and then preforms the task(s).

To implement DCI in PHP we can:

  1. Develop a base Data class that provides data storage and primitive getters and setters.
  2. Develop a Context interface that defines methods for assigning/unassigning roles as needed.
  3. Implement the Context interface in a ContextualData class, using __call() to execute the injected methods.

Implementing DCI in PHP

The Data Class

class Data
{
    protected $_data;

    public function __construct($data)
    {
        $this->_data = $data;
    }

    public function __get($name)
    {
        if (array_key_exists($name, $this->_data)) {
            return $this->_data;
        } else {
            return null;
        }
    }

    public function __set($name, $value)
    {
        $this->_data[$name] = $value;
    }
}

The basic Data class meets the requirements set forth in the first rule: data can be accessed to either read or write, but that's it.

The Context Interface

interface Context
{
    public function hasRole($role);

    public function assign($role, $methods);

    public function unassign($role);

    public function unassignAll();
}

The Context interface defines basic functionality to:

  • determine whether a given role as been assigned
  • assign a role
  • unassign a role
  • unassign all roles (remove all context

Implementing the Context Interface

Now comes the actual logic necessary to implement DCI in PHP:

class ContextualData extends Data implements Context
{
    protected $_roles;

    public function __construct($data)
    {
        parent::__construct($data);
        $this->_roles = array();
    }

    public function __call($function, $args)
    {
         foreach ($this->_roles as $role)
         {
             if (array_key_exists($function, $role)) {
                 $func_args = array_merge(array('data' => $this), $args);
                 return call_user_func_array($role[$function], $func_args);
             }
         }
    }

    public function hasRole($role)
    {
         return array_key_exists($role, $this->_roles);
    }

    public function assign($role, $methods)
    {
        $this->_roles[$role] = $methods;
    }

    public function unassign($role)
    {
        unset($this->_roles[$role]);
    }

    public function unassignAll()
    {
        $this->_roles = array();
    }
}

In this implementation, a role is a named array using the method names as the keys and callable object (e.g. anonymous functions) as the keys. To assign a role, you pass the role name and the role array to the assign() method. To assign a role, you simply pass the role name to unassign().

Wrapping Up

Now that we have the data and context taken care of, let's wrap up with an example of interaction using the cliché (if not classic) use case of transferring money between two bank accounts:

// Instantiate two accounts
$account1 = new ContextualData(array('balance' => 100));
$account2 = new ContextualData(array('balance' => 25));

// Define the source and destination account roles
$source_account = array(
        'withdraw' => function($data, $amount) {
                $data->balance = $data->balance - $amount;
            }
);
$dest_account = array(
        'deposit' => function($data, $amount) {
                $data->balance = $data->balance + $amount;
            }
);

// Assign the roles and execute the transfer
$account1->assign('SourceAccount', $source_account);
$account2->assign('DestAccount', $dest_account);
$account1->withdraw(50);
$account2->deposit(50);
echo "Account 1 balance is: " . echo $account1->balance . PHP_EOL;
echo "Account 2 balance is: " . echo $account2->balance . PHP_EOL;

The output of our script would be:

Account 1 balance is: 50
Account 2 balance is: 75

Thoughts?

Like I said in my disclaimer, I'm not really claiming that this is the best way to implement DCI in PHP. I just wanted to get my own thoughts down somewhere and, perhaps, get some feedback from others.

Understanding Javascript Events and Promises

Earlier today a friend of mine asked me to explain events in Javascript. I took a quick stab at it, but despite him being one of the smartest people I know, he claims I did a really shitty job and left him more confused than anything else. Challenge excepted, Dan.

Why We Need Events and Promises

To grasp what events and promises are, it's helpful to understand the problems they solve. So let's start there.

Events

Javascript was developed to allow Web developers to create dynamic and interactive websites. To do that, developers needed a way to say, "when this happens (the event), do that."

For example, consider a something fairly simple like form validation. The goal is to execute some code after the user clicks the submit button but before the form submission actually occurs (perhaps even preventing that from happening). So, we create some code to listen for the submission event and respond to it:

var validateForm = function() {
  // Validation code goes here

 // Return true if valid, return false if not
};

<form onsubmit="return validateForm();">
  <!-- form elements go here -->
</form>

The form element's "onsubmit" attribute is executed when the user submits the form but before the submission is actually performed. By returning false when the form does not validate, you're telling the browser to not submit the form as it normally would. Returning anything else tells the browser to carry on as planned.

So the basic idea is this: You want your code to execute when a particular event occurs, but you have no idea if/when the event will occur. So you tell the browser to "listen" for the event (in this case, when our form is submitted) and execute your "callback" when it does.

Promises

Promises take the idea of events a little bit further (and obscure them with a new name just to keep it interesting).

One of the primary purposes of employing Javascript is presenting a snappy and fluid user experience, but the Internet is not always snappy and is rarely fluid. (Just think about how annoying it would be if you had to reload Gmail every time you wanted to check for new messages). So to compensate for that fact, Javascript is asynchronous.

In other words, when you instruct the browser to do certain tasks, your code continues to execute beyond the request as if it were instantly handled. In reality, it's being completed in the background. But that poses a problem, what do you do if you need to know the outcome of the request to decide what you want to do?

Think about the Gmail example. We don't want everything to stop while we check the server for new messages, but we also need to make sure to do something if new messages do exist.

var checkmail = function() {

    // First we define a function to handle the response
    var parseResponse = function(eventObject, requestObject) {
        if (requestObject.response.messageCount > 0) 
            // Process the new msessages
        }
    };

    // Then we execute the request and pass a reference to our function
    var request = new jQuery.post('http://myserver.com');
    request.ajaxSuccess(parseResponse);
};

Okay, to be honest, I've simplified this a little bit, but the concepts are all there. Here is the basic idea: We define a request to execute and attach a listener to execute when the request is successfully completed (this is the promise---the request object is promising to execute our code when it's completed the request).

Building On the Basics

Okay, so we bind to events and contract with promises using anonymous functions (or other callables). When the event occurs or the promise is fulfilled, our function executes. That may not seam terribly powerful, but when you consider that scope is passed with the callable and that arguments can be passed to the callable, the real power is revealed:

  • When an anonymous function is bound to a DOM element's event, the function executes in the element's context (the this keyword references the element emitting the event.
  • Arguments can be used to pass information about the event when it occurs or the promise is fulfilled (e.g. the payload of an AJAX response, a status code, or a even another promise).

In Conclustion

So those are the basics, but things can get much more complicated. Still, if you keep all of this in mind, you should be able to grasp the more advanced concepts with out too much trouble.

Here are some more advanced implementations of events and promises:

  • external libraries (e.g. jQuery) offer normalized APIs for binding with and responding to window, document and element events
  • HTML websockets provide various promises that can be used to develop unique and awesome Web applications over custom (or at least non-HTTP) protocols
  • non-browser platforms (including operating systems such as Windows 8 and desktop applications like GNOME) provide advanced asynchronous APIs that make it easy to write fully fledged applications using javascript, HTML and CSS

Adding Capabilities to Default WordPress Roles

WordPress includes several default user roles, each with its own set of capabilities. These capabilities are the cornerstone of permissions in WordPress and provide easy privilege checking:

if (current_user_can('do_some_capabilitiy')) {
    // Do something that only certain users can do
}

The default capabilities are used by WordPress itself to control which settings and functions each user has access to. Only administrators, for example, can change theme options—which include background images, header images, widgets, etc.

Sometimes the defaults don't quit fit, though, and you'll want to add just one or two permissions to a role that doesn't have them without granting the whole slew of capabilities provided by the next role up.

Adding capabilities to default roles

The easiest way to add capabilities to a default role is through the user_has_cap filter, which is called whenever a capability test is performed (e.g. when current_user_can() is called).

The following example demonstrates how to give editors the ability to change theme options:

function modify_user_capabilities($capabilities, $capability)
{
    switch ($capability[0]) {
        case 'edit_theme_options':
            // Allow editors to change theme options
            if ((!array_key_exists('edit_theme_options', $capabilities) || !$capabilities['edit_theme_options']) && array_key_exists('editor', $capabilities) && $capabilities['editor'] == 1) {

                 $capabilities['edit_theme_options'] = true;
            }
    }
    return $capabilities;
}
add_filter('user_has_cap', 'modify_user_capabilities', 10, 2);
Look here for more information on WordPress filters.

Does PHP autoloading work with Exception classes?

I've noticed a lot of frameworks and SKDs are very good about calling required_once() to load a custom exception class before actually throwing the exception, which begs the question, "Can't autoloading take care of that?" I did a little investigating, and the answer is: A little yes; a little no.

Exception Classes Do Get Autoloaded

Yes. Exception classes will get autoloaded when they're thrown:

function autoload($classname)
{
 echo "autoload called\n";
 class MyException extends Exception{}
}

spl_autoload_register('autoload');

try {
 echo "throwing exception\n";
 throw new MyException();
 catch (MyException $ex) {
 echo "exception caught\n";
}

The previous code will output:

throwing exception
autoload called
exception caught

So exception classes will indeed be autoloaded. So why do major frameworks (e.g. Zend) and SDKs (e.g. Amazon) always include a require_once() call to ensure the class is loaded? Because there's no guarantee the autoloading mechanism is their own.

The Zend Framework loves using its own autoloading classes, but it's designed to be used in pieces and cannot be certain (1) any autoloading will actually be enabled or (2) the autoloading that is enabled will know where to find its exception classes (they're PSR-0 compliant, but still . . .).

So what's best practice?

In my opinion, you can feel free to rely on autoloading whenever you will always control the environment your code executes in. Otherwise, don't rely on autoloading (and that goes for all classes, not just custom exceptions).

What do you think?