Custom Data Transactions in Laravel Lumen
Before we begin: This article assumes you have at least a moderate working knowledge of developing an API with Lumen. As such I will be skipping over basic steps like creating a database migration and other tasks you should already be familiar with if you’re reading an article like this.
In developing an API based on the Laravel Lumen framework I ran into a situation where I wanted to selectively log the data being changed by an API call. In short, I wanted to know the record, the attribute, the old data, and the new data along with the timestamp of when the change was made and store it in my database. I wanted this data for audit purposes on this project. I also wanted control of this functionality and to not log changes on every call, just selected calls. The idea was I would be able to expose this data via a secured call to a client support application.
So, let’s dive in to how I tackled this problem. First, I laid out how I wanted to store the data in my database. My thought process largely went:
- I want to be able to call up all the changes for a single transaction (API call).
- I want each data point change logged separately so that I’m not storing all the changes as a serialized array or something on the transaction.
- I wanted to know the type of transaction (new record, update, deletion).
This led me to a two-table setup. One table for the transaction that stored the event date, the transaction type, the name of the table altered by this transaction, and the primary key of the altered record. The ID on this table would be used as a foreign key on the table that stored each of the changed items in that transaction in separate records. This would be the transaction items table. It would store the transaction id (from the transactions table), the column name, the old data and the new data.
So create a new migration for a table called ‘data_transactions’. Mine looks like this:
public function up() {
Schema::create(
'data_transactions',
function (Blueprint $table) {
$table->increments('id');
$table->timestamp('event_date')->default(DB::raw('CURRENT_TIMESTAMP'));
$table->string('transaction_type');
$table->string('table_name');
$table->integer('primary_key');
}
);
}It’s important to note that I also have a bunch of indexes being created in my live code that I have omitted here in the interests of clarity.
Now let’s create our ‘data_transactions_items’ table:
public function up() {
Schema::create(
'data_transactions_items',
function (Blueprint $table) {
$table->bigIncrements('id');
$table->integer('transaction_id')->unsigned();
$table->foreign('transaction_id')->references('id')->on('data_transactions');
$table->string('column_name');
$table->longText('old_data')->nullable();
$table->longText('new_data')->nullable();
}
);
}Once you’re happy, go ahead and run the migration to create these tables. Now we’ll need to create basic models for each of these tables. You’ll be able to extend these later but let’s add them to ‘app/Models’ or wherever you keep your models.
class DataTransactions extends Model {
protected $fillable = ['event_date','table_name','transaction_type','primary_key'];
public $timestamps = false;
}And for ‘data_transactions_items’…
class DataTransactionsItems extends Model {
protected $fillable = ['column_name','old_data','new_data'];
public $timestamps = false;
}You may have noticed the “transaction_type” column on the “data_transactions” table. This refers to application constants I created and mentioned earlier to indicate the type of the transaction:
if (!defined("DATA_TRNSCTN_NEW")) {
define("DATA_TRNSCTN_NEW",1);
}
if (!defined("DATA_TRNSCTN_UPDATE")) {
define("DATA_TRNSCTN_UPDATE",2);
}
if (!defined("DATA_TRNSCTN_DELETE")) {
define("DATA_TRNSCTN_DELETE",3);
}My next step was to create a Trait that I could bring into any class that I wanted to use this feature on. I like PHP traits to keep reusable, generic functions that can be used to extend the functionality of a class. I keep all my traits in ‘app/Traits’, but you can set up your own hierarchy that works for you. Here’s my ‘DataTransaction’ trait:
<?phpnamespace App\Traits;use App\Events\DataTransactionEvent;trait DataTransaction {protected function _fire_data_transaction_event($dc,$tx_typ,$old_data,$new_data) {
$tx['table_name'] = $dc->getTable();
$tx['transaction_type'] = $tx_typ;
$tx['primary_key'] = $dc->id;
$tx['old_data'] = $old_data;
$tx['new_data'] = $new_data;
event(new DataTransactionEvent($tx));
}
}
So we have a single function here called “_fire_data_transaction_event” that takes all the transaction data, packs it up in an array and then sends it off to Lumen’s built-in event listener for evaluation and logging. But first, let’s look at how “_fire_data_transaction_event” is called in a controller.
We’ll bring the trait into our Controller:
use App\Traits\DataTransaction;Next, declare it as we create the class:
class UsersController extends Controller {
use DataTransaction;In the case where we’re creating a new user there will be no old data, and we’ll make sure $tx_type is the DATA_TRANSACTION_NEW constant. I’m going to skip over the basic stuff here like setting up the data to be sent to your Model. So we bring in the User model as $user.
$user = new Users;Now, after we have saved the record and confirmed it was successful we can call our “_fire_data_transaction_event” method from our trait:
$user->save();
if ($user->id) {
$this->_fire_data_transaction_event($user,DATA_TRNSCTN_NEW,array(),$user->toArray());
}So we’re passing the $user model, the new transaction constant, a blank array for $old_data, and an array of the model data for $new_data.
There’s a slight difference calling this on the method that updates an existing record. Let’s look at that:
$user = Users::get_user($key);
$current = $user->toArray();In this example we load the user from our model and then store the data on the record as an array called $current. This is effectively our $old_data for the transaction trait. Now, when we call the event:
if ($user->save()) {
$this->_fire_data_transaction_event($user, DATA_TRNSCTN_UPDATE, $current, $user->toArray());
}Again we pass the model, the transaction type constant, $current as our $old_data, and the modified data as the $new_data.
At this point, I just want to call back to the trait to point out something.
$tx['table_name'] = $dc->getTable();We use a built-in function called getTable() that exists on the passed model object to get the name of the table the model references. That’s how that magic works. OK, let’s continue and build the event and event listener.
In ‘app/Events’ create a new event class called ‘DataTransactionEvent’. This event is going to do some analysis. For example, if a column is unchanged we don’t want to log it. So we’ll need to run a compare on the new data and the old data and drop off items that shouldn’t be logged. We will also have an array of excluded columns that will be removed immediately. These are things like default columns like id, created_date, updated_date, etc. We don’t care if those items are changed or not.
<?phpnamespace App\Events\Core;use App\Events\Event;class DataTransactionEvent extends Event {
public $data;
private $_tx_typ;
private $_old_data;
private $_new_data;public function __construct($data) {
$this->_old_data = $data['old_data'];
$this->_new_data = $data['new_data'];
$this->_tx_typ = $data['transaction_type'];
$this->_compare();
$data['old_data'] = $this->_old_data;
$data['new_data'] = $this->_new_data;
$this->data = $data;
}private function _compare() {
switch ($this->_tx_typ) {
case DATA_TRNSCTN_NEW:
case DATA_TRNSCTN_UPDATE:
foreach ($this->_old_data as $key => $value) {
if (array_key_exists($key, $this->_new_data)) {
if ($this->_new_data[$key] == $this->_old_data[$key]) {
// Data is the same, unset the array key as we don't need it
unset($this->_old_data[$key]);
unset($this->_new_data[$key]);
}
}
}
break;
case DATA_TRNSCTN_DELETE:
foreach ($this->_old_data as $key => $value) {
$this->_new_data[$key] = null;
}
break;
}
}
}
You should be familiar with how Lumen events work. When called by “_fire_data_transaction_event” the data array that function created is intercepted by the __construct() method on the event class. I’m translating them to class variables to make handling the data easier but that’s just a personal style choice. Then I send it off to the compare() function. This is the function that takes care of dropping off items we don’t need to log (unchanged values). You’ll note I switch based on the transaction type because for a delete I want to save an image of all the data that existed in that record before it was deleted. This all saved in the class-level copies of the arrays that I made earlier.
Finally, the data comes back and we update the passed $data object with the array values that need to be logged.
Next, we create our listener in ‘app/Listeners’. We will call ours ‘DataTransactionWrite’ as this is where our transaction is logged to the database.
<?phpnamespace App\Listeners\Core;use App\Events\Core\DataTransactionEvent;
use App\Models\Core\DataTransactions;
use App\Models\Core\DataTransactionsItems;class DataTransactionWrite {public function handle(DataTransactionEvent $event) {
if (!empty($event->data['new_data'])) {
$tx = new DataTransactions;
$tx->table_name = $event->data['table_name'];
$tx->primary_key = $event->data['primary_key'];
$tx->transaction_type = $event->data['transaction_type'];
if ($tx->save()) {
foreach ($event->data['new_data'] as $key => $value) {
$tx_itm = new DataTransactionsItems;
$tx_itm->transaction_id = $tx->id;
$tx_itm->column_name = $key;
if (array_key_exists($key,$event->data['old_data'])) {
$tx_itm->old_data = $event->data['old_data'][$key];
}
$tx_itm->new_data = $value;
$tx_itm->save();
}
}
}
}
}
This should be pretty self-explanatory, we’re creating the transaction record first and once we have it’s primary key we iterate through every ‘new_data’ value to create a transaction item record. To avoid errors we only add the old_data value if it exists in that event data array.
Finally, to tie this all together we have to tell the EventServiceProvider.php file in ‘app/Providers’ to listen for our event and hand it off to the listener class. That’s done in the $listen array:
protected $listen = [
'App\Events\Core\DataTransactionEvent' => ['App\Listeners\Core\DataTransactionWrite']
];If you’ve done everything correct, making an API call to whatever controller method you attached _fire_data_transaction_event() to you should log that transaction in your database.
And if we go to data_transactions_items and look for the records for transaction id 34 we see the three values that were changed and the old and new data:
Now we can track changes to critical data in our API and have a way of then making them available to people who might need to check this data via our very same API.
I hope this article helped you with a dilemma you were facing. I hope to see you again as we explore other cool stuff!
