Reflection on life

It has been awhile since my last technical posting, a reflection of my current situation as a middle management.

Most technologist in Asia or at least in my narrow perspective could choose to either become an expert in a certain field and risk becoming obsolete or pursue a life trying to scale the corporate ladder.

I have chosen the latter and what it means is a shift in focus from being the problem solver to becoming a shape-shifter taking on the role of a saleman, slave driver, morale booster, leader, prophet, jester.

The role of a manager can be rewarding only if you are given the autonomy to lead your team and that you can produce results. And it is only fair that you should take responsibility of your team's performance.

But often time you will need to do the bidding of higher-ups whose direction you might not always agreed with. So how do you work on something that even you do not believe in ?? It is not impossible, but just very unfulfilling. How do you escape this cycle??

After going through some unfulfilling years at work, i have decided to take a year break from it all. An opportunity to slow down and appreciate life. A quest to do some reflection, to recharge my mind, body and soul (MBS).

As i struggle to define what is MBS, i guess the easiest approach would be to add activities that falls into this category in an iterative manner. The list would be flexible and dynamic but this post will be a good place to note down my commitment.

Mind
Learn something new - (Doing my Masters in information security)
Learn a new language

Body
Exercise - at least twice a week either swim, gym, jog or hike
Sleep - at least 8 hrs
Eat well 

Soul
Practice mindfulness - need to learn that first
Meet new people
Chill 

php imap download email to eml a comparison of different approaches

Pieced together a simple php script to download emails using imap.

These are some of the different approaches to download the email as eml format. 

 Approach A   
 $headers = imap_fetchheader($connection, $k, FT_PREFETCHTEXT);  
 $body = imap_body($connection, $k);  
 file_put_contents("pathtoemail.eml", $headers . "\n" . $body);  
 Approach B  
  $body=imap_fetchbody($connection, $k, "");  
 file_put_contents("pathtoemail.eml", $body);  
 Approach C  
  imap_savebody($connection, "pathtoemail.eml", $k);  

Performance comparison when downloading 1 day worth of email. (lower is better)
 Approach A (mins)  
 1. 2.095  
 2. 1.886  
 3. 1.984  
 4. 1.990   
 Approach B (mins)  
 1. 1.447  
 2. 1.357  
 3. 1.399  
 4. 1.413  
 Approach C (mins)  
 1. 1.468  
 2. 1.409  
 3. 1.401  
 4. 1.367  


Conclusion:

Either approach B or C would make a good choice for downloading emails using php imap.

Sample script:

 <?php  
 $time_start = microtime(true);   
 //$server = '{imap.mail.yahoo.com:993/ssl}';  
 $server = '{imap.gmail.com:993/ssl}';  
 //$user = 'xxx@gmail.com';  
 $user = $argv[1];  
 //$pass = 'xxx';  
 $pass = "$argv[2]";  
 date_default_timezone_set('Asia/Shanghai');  
 $criteria = 'SINCE "'.date('d M Y', strtotime('- 1 days')).'"';  
 $connection = imap_open($server, $user, $pass) or die("can't connect: " . imap_last_error());  
 $mailboxes = imap_list($connection, $server, '*');  
 $folder = "./$user/";  
 if(!file_exists($folder)){  
   mkdir($folder, 0777, true);  
 }  
 foreach($mailboxes as $mailbox) {  
   $shortname = str_replace($server, '', $mailbox);  
   echo "$shortname\n";  
   //create folder  
   if(!file_exists("$folder$shortname")){  
     mkdir("$folder$shortname", 0777, true);  
     }  
   //enter into folder  
   imap_reopen($connection, $server.$shortname);  
   $count = imap_num_msg($connection);  
   $uids = imap_search($connection, $criteria, SE_UID, 'UTF-8');  
   if(is_array($uids)){  
     foreach($uids as $ud){    
       $k = imap_msgno ($connection, $ud);  
       //approach A  
       //$headers = imap_fetchheader($connection, $k, FT_PREFETCHTEXT);  
       //$body = imap_body($connection, $k);  
       //file_put_contents("$folder$shortname/$ud.eml", $headers ."\n". $body);  
       //approach B  
       //$body=imap_fetchbody($connection, $k, "");  
       //file_put_contents("$folder$shortname/$ud.eml", $body);  
       //approach C  
       imap_savebody($connection, "$folder$shortname/$ud.eml", $k);  
       echo "saved $ud.eml " . "($count-$k)\n";    
     }  
   }else{  
 //          echo "imap_search failed: " . imap_last_error() . "\n";  
     }  
 }  
   imap_errors();  
   imap_close($connection);  
 $time_end = microtime(true);  
 //dividing with 60 will give the execution time in minutes other wise seconds  
 $execution_time = ($time_end - $time_start)/60;  
 //execution time of the script  
 echo 'Total Execution Time: '.$execution_time.' Mins';  
 ?>  

Starting a VPS reseller business

Just some notes to remind me on how to setup my own VPS services

Pre-requisite:

1 x Server grade dedicated server
Multiple number of IP addresses; 1 for each VPS.
Visualization software:
a) http://solusvm.com/pricing/
b) http://www.virtualizor.com/NOCs

Payment portal
a) WHMCS
b) http://www.docs.modulesgarden.com/SolusVM_Extended_VPS_For_WHMCS

Manually setup Wifi AP with a captive portal on kali linux

As per the topic, this is a post to show how to create a (i) Wifi AP, (ii) redirect users to a captive portal, and (iii) how to allow user to use the internet after they accept the terms and condition on the captive portal.

This is the environment we will be using:
Local interface which is connected to the internet.

1)
interface configuration are located here /etc/network/interfaces
eth0
IP : 192.168.17.28
Mask: 255.255.255.0
GW : 192.168.17.2

Wireless Network
wlan0
IP : 192.168.0.1
Mask: 255.255.255.0
GW : 192.168.0.0

Assuming you already have eth0 setup properly, turn on wireless network interface

 ifconfig wlan0 192.168.0.1  

2)
Configure DHCPD /etc/dhcp/dhcpd.conf

      default-lease-time 300;  
      max-lease-time 360;  
      ddns-update-style none;  
      authoritative;  
      log-facility local7;  
      subnet 192.168.0.0 netmask 255.255.255.0 {  
      range 192.168.0.100 192.168.0.200;  
      option routers 192.168.0.1;  
      option domain-name-servers 192.168.17.2;  
      }  

3)
Enable IP forwarding

 echo "1" > /proc/sys/net/ipv4/ip_forward  

4)
Flush your iptables rules so that we are on the same settings

 iptables -t nat -F  

5)
Configure Iptables to enable traffic masquerading

 iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE   

6)
Configure all Wifi traffic from wlan0 to be redirected to the webserver

 iptables -t nat -A PREROUTING -i wlan0 -p tcp -j DNAT --to-destination 192.168.17.128:80  

7)
Configure your hotspot configuration. HOSTAPD is a utility for creating a AP.
create your configuration file /etc/hostapd/hostapd.conf

# Define interface  
interface=wlan0
driver=nl80211
# Select driver  
ssid=myhotspot
# Set access point name  
hw_mode=g
# Set access point harware mode to 802.11g  
# Enable WPA2 only (1 for WPA, 2 for WPA2, 3 for WPA + WPA2)  
# Set WIFI channel (can be easily changed)  
channel=6
#wpa=2
#wpa_passphrase=mypassword

8)
Time to start DHCPD and hostapd. once started use another device to see if there is a myhotspot.  Everytime you try to surf the internet, you will be redirected to your localhost webserver. Whatever your are serving is called the captive portal.

 dhcpd  
 hostapd /etc/hostapd/hostapd.conf  
 /etc/init.d/apache2 start  

9)
So how do you enable internet for users? One possible way is to use your captive portal to create Iptable rule which enable a particular device (mac address). For example if your device is 11:22:33:11:22:33

 iptables -t nat -I PREROUTING -m mac --mac-source 11:22:33:11:22:33 -j ACCEPT  

Still unclear on step 9? Try looking at pwnstar package.

Having trouble routing ur wifi to ur internet interface ?
https://prahladyeri.wordpress.com/2013/05/26/how-to-turn-your-linux-machine-into-a-wifi-access-point/

Google Scraper

I am in need for analyzing google search result, fortunately there are multiple opensource solution out there. But google hates scrapers and would block your IP should they determine that you are breaking their terms and condition. 

Possible Google Scraper: (Play with the sleep timing between request to prevent IP blocking)

https://github.com/NikolaiT/GoogleScraper

https://github.com/MarioVilas/google

//Example using MarioVilas's google scraper: 

python google.py --stop=20 "inurl:console filetype:php" > test.txt

//If you need to remove parameters, a simple bash script is perfect: 

vi removeparameter.sh

#!/bin/bash

while read p; do

FILE=$p

echo ${FILE%%\?*}

done < test.txt

Installing Evilgrade on Centos 6

Installing Evilgrade on Centos gave me abit of problem and took too much time for me thus i am documenting it.

Download Evilgrade from:
https://github.com/infobyte/evilgrade/archive/master.zip

unzip master.zip and go to the directory.

//if you have no error then gd else you will need to install the dependencies
./evilgrade

Required perl dependencies:

Data::Dump
Digest::MD5   
Time::HiRes
RPC::XML

How to install them on Centos using CPAN:
use CPAN, but if you have never used it, then you will need to configure it.

perl -MCPAN -eshell
//step thru the configuration

// key in the below command to automate the configuration
o conf init

cpan -i Data::Dump
cpan -i Digest::MD5
cpan -i Time::HiRes
cpan -i RPC::XML

if you have error with RPC::XML, 
yum whatprovides "perl(XML::Parser)"
yum install ""

yum install perl-IO-Socket-SSL

Setting up Wordpress on VPS

This is just to journal down the steps required for setting up a fresh installation of Wordpress on a VPS. FTP account is required and i assumed that your VPS has it.

Install mysql and mysqld

 yum install mysql mysqld  

Install PHP

 yum install php php-mysql  

Install Wordpress

 wget http://wordpress.org/latest.tar.gz  
 tar -xzvf latest.tar.gz   
 cp ./wordpress/wp-config-sample.php ./wordpress/wp-config.php  
 sudo cp -r ./wordpress/* /var/www/html  
 sudo yum install php-gd  
 sudo service httpd restart  

Install Mysql

sudo yum install mysql-server   
sudo service mysqld start   

sudo /usr/bin/mysql_secure_installation   

  Enter current password for root (enter for none):    
  OK, successfully used password, moving on...   
  By default, a MySQL installation has an anonymous user, allowing anyone   
  to log into MySQL without having to have a user account created for   
  them. This is intended only for testing, and to make the installation   
  go a bit smoother. You should remove them before moving into a   
  production environment.   
  Remove anonymous users? [Y/n] y              
  ... Success!   
  Normally, root should only be allowed to connect from 'localhost'. This   
  ensures that someone cannot guess at the root password from the network.   
  Disallow root login remotely? [Y/n] y   
  ... Success!   
  By default, MySQL comes with a database named 'test' that anyone can   
  access. This is also intended only for testing, and should be removed   
  before moving into a production environment.   
  Remove test database and access to it? [Y/n] y   
  - Dropping test database...   
  ... Success!   
  - Removing privileges on test database...   
  ... Success!   
  Reloading the privilege tables will ensure that all changes made so far   
  will take effect immediately.   
  Reload privilege tables now? [Y/n] y   
  ... Success!   
   $   

Create Mysql database

mysql -u adminusername -p  
CREATE DATABASE databasename;  
GRANT ALL PRIVILEGES ON databasename.* TO "root@localhost" IDENTIFIED BY "p@ssw0rd";  
FLUSH PRIVILEGES;     

Configure Mysql and link it with Wordpress wp-config.php

 define('DB_NAME', 'wordpress');  
 define('DB_USER', 'user');  
 define('DB_PASSWORD', 'password');  

Try surfing to Wordpress Site to complete the installation

 serverip/wp-admin/install.php  

Next configure so that themes can be installed automatically

 //add this line of code to wp-config.php  
 if(is_admin()) {  
      add_filter('filesystem_method', create_function('$a', 'return "direct";' ));  
      define( 'FS_CHMOD_DIR', 0751 );  
 }  

Some bugs with WP meant that inorder to create folders automatically i had to edit permission to WP-Content folder

 chmod 777 -R /var/www/html/wp-content  

References
http://www.wpmayor.com/the-ultimate-guide-to-setting-up-a-wordpress-vps-part-1/
https://www.digitalocean.com/community/tutorials/how-to-install-wordpress-on-centos-6--2
http://www.wpbeginner.com/wp-tutorials/how-to-fix-the-error-establishing-a-database-connection-in-wordpress/
http://wordpress.org/support/topic/your-php-installation-appears-to-be-missing-the-mysql-extension-which-is-require