Wednesday, December 31, 2014

Enable SMTP on max OS

sudo postfix start

java send mail using ubuntu local sendmail smtp

import java.util.Properties;

import javax.mail.Message;
import javax.mail.MessagingException;
import javax.mail.Session;
import javax.mail.Transport;
import javax.mail.internet.AddressException;
import javax.mail.internet.InternetAddress;
import javax.mail.internet.MimeMessage;


public static void main(String[] args) {

String to="recipient@example.com";
String from="sender@example.com";

Properties props = new Properties();
Session session = Session.getDefaultInstance(props, null);

// props.put("mail.smtp.host", "localhost"); // optional

String msgBody = "Sending email using JavaMail API...";

try {
Message msg = new MimeMessage(session);
msg.setFrom(new InternetAddress(from, "NoReply"));
msg.addRecipient(Message.RecipientType.TO,
new InternetAddress(to, "Mr. Recipient"));
msg.setSubject("Welcome To Java Mail API");
msg.setText(msgBody);
Transport.send(msg);
System.out.println("Email sent successfully...");

} catch (AddressException e) {
throw new RuntimeException(e);
} catch (MessagingException e) {
throw new RuntimeException(e);
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}

}

Wednesday, December 24, 2014

Thursday, December 18, 2014

Setup Zabbix on Ubuntu for Postgresql connection

1. Install unixODBC >sudo apt-get install unixODBC
2. Install postgresql odbc driver >sudo apt-get install postgresql-odbc
3. Setup /etc/odbc.ini
[test]
Description = 192.168.8.220 test
Driver = postgresql
Servername = 192.168.8.220
Port = 5432
Database = db_name

4. Setup /etc/odbcinst.ini
[postgresql]
Description = ODBC for postgresql
Driver = /usr/lib/x86_64-linux-gnu/odbc/psqlodbca.so
Setup = /usr/lib/x86_64-linux-gnu/odbc/libodbcpsqlS.so

Test:
>isql test your_name your_password -v

In Zabbix, add database monitor:
key = db.odbc.select[some_name,test]

Postgresql usage notes

Revoke:

    REVOKE ALL ON DATABASE db_name FROM user_name;
    revoke all on all tables in schema public from zabbix;

Readonly User:

    revoke all on schema public from public;
    create role zabbix login password 'some_pass';
    grant usage on schema public to zabbix;
    grant select on public.some_table to zabbix;


When you create a new database, any role is allowed to create objects in the public schema. To remove this possibility, you may issue immediately after the database creation:

REVOKE ALL ON schema public FROM public;

Edit: after the above command, only a superuser may create new objects inside the public schema, which is not practical. Assuming a non-superuser foo_user should be granted this privilege, this should be done with:

GRANT ALL ON schema public TO foo_user;


Friday, December 12, 2014

Configure Zabbix to sent alert email

Setup the host to be monitored


Setup Media Type, in this case, use email and local "sendmail" smtp server.  Remember to use 127.0.0.1 on the SMTP server settings.  Otherwise, it may have connection problem.



Ensure the User media email is enabled.



Ensure action is enabled.




Set the trigger.





Thursday, December 11, 2014

Connects zabbix server to agent

* Remember to setup the zabbix-agent configuration to listen to the
right server IP and other settings.

log: /var/log/zabbix-agent/zabbix_agentd.log
config: /etc/zabbix/zabbix_agentd.conf

Check opened ports

netstat -anltp | grep "LISTEN"

ssh-copy-id copy id to remote machine

usage> ssh-copy-id -p 1337 your_name@remote_host

Wednesday, November 12, 2014

Monitor linux process memory or cpu resource usage

Find the process: > ps -ef | grep java

>top -p pid

interactive commands:
"f" - add/drop columns
"c" - command details
"M" - sort by memory
...

Wednesday, November 5, 2014

Tools: SAP Crystal Reports

SAP Crystal Reports

A robust production reporting tool, SAP Crystal Reports turns almost any data source into interactive, actionable information that can be accessed offline or online, from applications, portals and mobile devices.

Friday, August 29, 2014

Postgresql JDBC enum and stored procedure

Simple way to submit postgresql enum from JDBC: Use type casting.

CallableStatement cs = conn.prepareCall("{call
sp_dummy(?::enum_result)}");
cs.setString(1, jobResult.toString());
cs.execute();

Thursday, August 28, 2014

Maven simple java application

mvn archetype:generate -DgroupId={project-packaging} -DartifactId={project-name} -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false

Saturday, August 23, 2014

Work Breakdown Structure


Estimating using Work Breakdown Structure

Work Breakdown Structure (WBS) is probably how 98% of software development projects are estimated. WBS consists of someone sitting down and saying "Okay, how are we going to get this done?", then coming up with a long laundry list like this:
  1. Setting up Development, Test, and Production environments - 8 hours
  2. Creating the Login screen - 4 hours
  3. Creating the Landing Page - 2 days
  4. Developing the XYZ Page - 12 hours
  5. etc., etc., etc.
You then continue with a long laundry list of every task you can imagine, with each task having an associated estimate. At the end you add up the time for all the tasks, and you have your overall time estimate.
Or do you?

WBS accuracy

As a practical matter, most WBS estimates I've seen under-estimate the total time required by a factor of 1.25 to 2.5. That is, if someone tells me that they did a WBS estimate and came up with 100 hours, the real answer is typically 125 to 250 hours.
IFPUG data also supports this. Though I don't have the exact data with me, I do remember that their data shows that estimates created using WBS alone under-estimate project effort by an average of about 50%. The reality is that this ratio actually varies quite a bit from person to person: some developers over-estimate, some under-estimate, some are very close. The good news is that people are consistent, and if they usually under-estimate by a factor of 2.5 they tend to always estimate by a similar ratio, unless ...

Use feedback to improve WBS estimates

If these developers are made to always review their estimates I believe they will eventually get better. For instance, let's say a developer told me a project would take 100 hours, but it really took 250 hours. After the project I would sit down with the developer and say "Okay, you estimated 100 hours, but it really took 250 hours. Now, I'm not here to beat you over the head. In fact, we like you quite a bit and want you on all of our future projects. But we need to take a little time and see where this estimate was off, okay?"
You do this behind closed doors, because everybody I know feels vulnerable when they're estimating. This stuff is hard, and it's not an exact science, I know that. But if you don't go through this process people won't get better.
Getting back to the WBS estimate, let's imagine that we worked through this process and we came up with an estimate of 1,600 hours. Given our team of four developers, this equates to 400 hours per developer. At 35 hours per week this is about 11.4 weeks of development effort.

Don't forget acceptance testing

Historically speaking, the people that I've seen take the WBS effort do not include final acceptance testing in their estimates, so I'll assume that's the case here. I'll use the same ratio of development time to final acceptance testing time that I used before (a 3:1 ratio), so if development takes 11.4 weeks, acceptance testing should take an additional 3.8 weeks (and the 1,600 hour estimate needs to be increased to 2,133 hours).

WBS estimate

Therefore, WBS predicts a total development time of 2,133 hours, or 15.2 weeks.

Thursday, August 21, 2014

Jersey Maven setup Log4j2

1. In pom.xml, add:

<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.0.1</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.0.1</version>
</dependency>

2. Add log4j configuration file to path: src/main/webapp/WEB-INF/classes
(create the classes folder manually)

3. Code:

Logger logger = LogManager.getLogger();
logger.info("info!!!!!!!!!!!!!!");

Serialize Object to Json indented string with Jersey

1. pom.xml - Use Jackson
        <dependency>
            <groupId>org.glassfish.jersey.media</groupId>
            <artifactId>jersey-media-json-jackson</artifactId>
        </dependency>   

2. Code:

        ObjectMapper mapper = new ObjectMapper();
        mapper.enable(SerializationFeature.INDENT_OUTPUT);
        String rJson = "";
        try {
            rJson = mapper.writeValueAsString(request);
            System.out.println(rJson);
        } catch (JsonProcessingException e1) {
            // TODO Auto-generated catch block
            e1.printStackTrace();
        }

Wednesday, August 20, 2014

Tomcat 7 connection pool with JNDI datasource and postgreSQL

1. Tomcat's content.xml

<Context>    <Resource name="jdbc/postgres" auth="Container"            type="javax.sql.DataSource" driverClassName="org.postgresql.Driver"            url="jdbc:postgresql://127.0.0.1:5432/mydb"            username="myuser" password="mypasswd" maxActive="20" maxIdle="10"  maxWait="-1"/>  </Context>

2. Web application's web.xml

<resource-ref>   <description>postgreSQL Datasource example</description>   <res-ref-name>jdbc/postgres</res-ref-name>   <res-type>javax.sql.DataSource</res-type>   <res-auth>Container</res-auth>  </resource-ref>

2. Code

    Connection conn = null;
        try {
            InitialContext cxt = new InitialContext();
            DataSource ds = (DataSource) cxt.lookup( "java:/comp/env/jdbc/postgres" );
            conn = ds.getConnection();
           
                 ...

                ...   
                while(rs.next()) {
                ...
                ...

                }
                rs.close();
            }
            conn.close();
        } catch (NamingException e1) {
            e1.printStackTrace();
        } catch (SQLException e) {
            e.printStackTrace();
        };

Maven Postgresql JDBC

1. Add dependency in pom.xml. e.g.
http://mvnrepository.com/artifact/org.postgresql/postgresql/9.3-1100-jdbc41
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>9.3-1100-jdbc41</version>
</dependency>

2. Coding:

Connection conn = null;
try
{
Class.forName("org.postgresql.Driver");
String url = "jdbc:postgresql://THE_HOST/THE_DATABASE";
conn = DriverManager.getConnection(url,"THE_USER", "THE_PASSWORD");
Statement st = conn.createStatement();
ResultSet rs = st.executeQuery("SELECT id, subject, permalink
FROM blogs ORDER BY id");
while ( rs.next() )
{
Blog blog = new Blog();
blog.id = rs.getInt ("id");
blog.subject = rs.getString ("subject");
blog.permalink = rs.getString ("permalink");
listOfBlogs.add(blog);
}
rs.close();
st.close();

}
catch (ClassNotFoundException e)
{
e.printStackTrace();
System.exit(1);
}
catch (SQLException e)
{
e.printStackTrace();
System.exit(2);
}

Monday, August 18, 2014

System server requirements

System hardware requirements
  CPU File I/O Database I/O RAM
Management Web App
High Low High Mid
Scoring Web App
Low Low to Mid Mid Mid
Workers
High High  Mid High
PostgreSQL Mid High N/A High
DWH N/A N/A High N/A
ETL High Low to Mid Mid High
CRM Mid Low Mid Mid
Analytics Engine High Low Mid High















System Server Requirements
  File Server Database Server Web Server Processing Server
Functionalities SFTP, NFS DWH, Operation DB Web based application, CRM, … Workers, ETL, Analysis Engine, …
CPU Low (E3-1230v2 or existing) Mid to High (E5-2420 dual / E52620 dual) Mid (E3-1230v2/existing server) High (E3-1230v2/existing server)
RAM 16G or above 32G or above 16G or above 16G or above
RAID RAID 1/5/6 RAID 10 NO RAID / RAID 1 NO RAID
RAID Controller Dell H710p or equivalent Dell H710p or equivalent N/A N/A
HDD TBD 1TB x 4 500GB 500GB
Quantity x1 x1 xN xN
Public IPs x1 N/A x2 N/A
Region All
All
All
All
Scalability Add drive for increase storage capacity.  Can add new server as required. Single server can support up to 3 to 5 years.  Can add new server as required. Scale Horizontally by adding new server as required. Scale Horizontally by adding new server as required.
New Server Price 20K (Dell R220) to 40K (Dell R520) 50K+ HKD (Dell R520/R720) 12K HKD (Dell R220) 12K HKD (Dell R220)





Servers Allocation
Live site

File Server Database Server Web Server Processing Server

New server Dell R220/R520 New server Dell R520/R720 HK  

20K/40K+ HKD - Stage 2+ 50K+ HKD - Stage 2 existing server #1 - Management  

  Support both all regions
existing server #2 - Scoring
 

  Database for Management, Scoring, Worker, DWH, etc. Dell R210 - Workers, ETL, SFTP  

    UK  

    existing #3 - Management  

    existing #4 - Scoring  

=R220, R210 #2 in Stage 1 = existing servers, R210 and R220 servers in Stage 1 Dell R220 #2 (12K HKD - Stage 1) - Worker, SFTP =R210, R220 #2 in Stage 1





Standby site

File Server Database Server Web Server Processing Server

New server Dell R220 RAID 1 New server Dell R220 with RAID 1 HK  

~20K HKD - Stage 3 (* Use low to mid-range standby server) R220 #3 (12K HKD - Stage 1.5) - Management, Scoring, SFTP, …  

  ~20K HKD - Stage 3 1 x VM #1 (2K HKD - Stage 1.5) Worker  

    UK  

    R220 #3 (12K HKD - Stage 1.5) - Management, Scoring, SFTP, …  

=R220 #3, R220 #4, VM #1, VM #2 in Stage 1.5 =R220 #3, R220 #4, VM #1, VM #2 in Stage 1.5 1 x VM #2 (2K HKD - Stage 1.5) Worker =VM #1, VM #2 in Stage 1.5

Thursday, August 14, 2014

Linux, Ubuntu log files location

=> /var/log/daemon.log : Running services such as squid, ntpd and others
log message to this file

=> /var/log/dmesg : Linux kernel ring buffer log

=> /var/log/dpkg.log : All binary package log includes package
installation and other information

=> /var/log/faillog : User failed login log file

=> /var/log/kern.log : Kernel log file

=> /var/log/lpr.log : Printer log file

=> /var/log/mail.* : All mail server message log files

=> /var/log/mysql.* : MySQL server log file

=> /var/log/user.log : All userlevel logs

=> /var/log/xorg.0.log : X.org log file

=> /var/log/apache2/* : Apache web server log files directory

=> /var/log/lighttpd/* : Lighttpd web server log files directory

=> /var/log/fsck/* : fsck command log

=> /var/log/apport.log : Application crash report / log file

=> /var/log/messages : General log messages

=> /var/log/boot : System boot log

=> /var/log/debug : Debugging log messages

=> /var/log/auth.log : User login and authentication logs

Wednesday, August 6, 2014

Measure the bandwidth between 2 servers

Computer A, B:
> sudo apt-get install iperf

Computer A:
> iperf -s

Computer B:
> iperf -c address_of_computer_A

Ubuntu Server reconfigure timezone

>date
>more /etc/timezone
>sudo dpkg-reconfigure tzdata
>sudo service cron stop
>sudo service cron start

Wednesday, July 9, 2014

Google Map simple marker

<!DOCTYPE html>
<html>
<head>
<meta name="viewport" content="initial-scale=1.0, user-scalable=no">
<meta charset="utf-8">
<title>Geocoding service</title>
<style>
html, body, #map-canvas {
height: 100%;
margin: 0px;
padding: 0px
}
#panel {
position: absolute;
top: 5px;
left: 50%;
margin-left: -180px;
z-index: 5;
background-color: #fff;
padding: 5px;
border: 1px solid #999;
}
</style>
<script src="https://maps.googleapis.com/maps/api/js?v=3.exp"></script>
<script
src="http://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
<script>
var geocoder;
var map;
function initialize() {
geocoder = new google.maps.Geocoder();
var latlng = new google.maps.LatLng(-34.397, 150.644);
var mapOptions = {
zoom: 8,
center: latlng
}
map = new google.maps.Map(document.getElementById('map-canvas'),
mapOptions);
}

var address_array =
[
"Shop 3-5, G/F, Po Fung Building, 1-7 Hop Yick Road, Yuen Long ",
"Rm 401, 4/F, Sincere House, 83 Argyle Street, Mong Kok ",
"Shop M001, Long Ping Commercial Complex, Long Ping Estate, Yuen Long ",
"Shop G06, G/F, Spot, 48 Lung Sum Avenue, Shek Wu Hui, Sheung Shui ",
"Unit 3, 2/F, United Success Commercial Centre, 508 Jaffe Road, Causeway
Bay "

];


count = 0
function codeAddress() {
//var address = document.getElementById('address').value;
var address = address_array[count];
if(typeof(address) == 'undefined') {
alert("Map completed.");
clearInterval(intervalID);
return;
}
geocoder.geocode( { 'address': address}, function(results, status) {
if (status == google.maps.GeocoderStatus.OK) {
map.setCenter(results[0].geometry.location);
var marker = new google.maps.Marker({
map: map,
position: results[0].geometry.location
});
} else {
if (status == google.maps.GeocoderStatus.OVER_QUERY_LIMIT) {
//alert("OVER_QUERY_LIMIT, just press ok to continue :) ");
count--;
} else {
the_index = address.indexOf(',');
if(the_index > 0) {
address_array.push(address.substring(the_index + 1,
address.length));
}
//alert('Geocode was not successful for the following reason: '
+ status);
//var error_messages = $('#messages').html() + status + " --- "
+ address + "<br/>"
//$('#messages').html(error_messages);
}
}
});
count++;
}


google.maps.event.addDomListener(window, 'load', initialize);

$(document).ready(function() {
//var intervalID = window.setInterval(codeAddress(), 1000);
start_map();
});

</script>
<script>

var intervalID;

function start_map() {
intervalID = window.setInterval(function(){codeAddress()}, 1000);
}

</script>

</head>
<body>
<div id="panel">
<!--
<input id="address" type="textbox" value="Sydney, NSW">
<input type="button" value="Geocode" onclick="start_map()">
-->
</div>
<div id="map-canvas" ></div>
<div id="messages" style='font-family:arial;' >The following
addresses cannot be marked: <br/></div>
</body>
</html>

Monday, June 23, 2014

Rails - Make Delayed job work with Rspec in tests

Turn off the queuing and let delayed job to run:

Delayed::Worker.delay_jobs = false

Friday, June 20, 2014

Rails override Time.now to return Time.zone.now

In initializer, add this:


class << Time
alias :system_now :now
def now
ActiveSupport::TimeWithZone.new(self.system_now.utc, Time.zone)
end
end

Tuesday, June 17, 2014

Nagios create plugins

On monitoring host:

Create plugin in /usr/lib/nagios/plugins (in C/Perl/Script/etc)
Enable plugin by:
1. define comment in /etc/nagios-plugins/config/somefile.cfg
2. edit /etc/nagios/conf.d/localhost_nagios2.cfg

Restart:
sudo service nagios3 restart

Thursday, June 12, 2014

Reviewed Ubuntu Server monitoring tools

Tried:
1. Nagios
2. Munin
3. Cacti
4. Zabbix

Turn out I like both Nagios and Zabbix.

Wednesday, June 11, 2014

Setup Nagios on Ubuntu 12.10

sudo apt-get update
sudo apt-get install nagios3 nagios-nrpe-plugin

If you have other web pages or apps running on the same server, the following information might be useful to separate Nagios as it's own Apache virtual host on a non-standard port (for example 43326 here).

Remove standard config and create Apache virtual host;

sudo cp /etc/apache2/conf.d/nagios.conf /etc/apache2/sites-available/nagio  s3  sudo rm /etc/apache2/conf.d/nagios.conf  sudo vi /etc/apache2/sites-available/nagios3  

To beginning of file add;

Listen 43326  <VirtualHost *:43326>      ServerAdmin webmaster@localhost      DocumentRoot /var/www/nagios3  

To the end of the file add;

</VirtualHost>  

Enable the new site;

sudo a2ensite nagios3  

If you run a firewall (UFW), open the port;

sudo ufw allow 43326  

Restart Apache

sudo service apache2 restart

Ubuntu OS upgrade

http://www.cyberciti.biz/faq/howto-upgrade-to-ubuntu-14-04-from-ubuntu-13-10-or-12-04/

Linux system monitoring

http://www.linuxscrew.com/2012/03/22/linux-monitoring-tools/
http://www.tecmint.com/command-line-tools-to-monitor-linux-performance/
http://www.cyberciti.biz/tips/top-linux-monitoring-tools.html
http://www.thegeekstuff.com/2011/12/linux-performance-monitoring-tools/
http://www.opensourceforu.com/2013/12/look-top-three-network-monitoring-tools/

Wednesday, May 21, 2014

Rails: Specify Time.zone in delayed job

Add the following code to the project's config/initializers/delayed_job.rb

The method_missing chain will add the current Time.zone.name to the
function's argument.

The Time.zone.name will be popped by the PerformableMethod.perform chain.


Delayed::DelayProxy.class_eval do

# Embed the current time zone name to delayed job's arguments when
# adding delayed job from delay() method.
def method_missing_with_custom_time_zone(method, *args)
raise "Time zone is nil" if Time.zone.nil?
args << {:custom_time_zone => Time.zone.name}
result = method_missing_without_custom_time_zone(method, *args)
result
end
alias_method_chain :method_missing, :custom_time_zone

end

Delayed::PerformableMethod.class_eval do

# Read time zone from job's argument.
def perform_with_custom_time_zone
if self.args.last.is_a?(Hash) &&
!self.args.last[:custom_time_zone].nil?
Time.zone = args.pop[:custom_time_zone]
else
raise "Custom time zone is missing from the arguments."
end
perform_without_custom_time_zone
end
alias_method_chain :perform, :custom_time_zone

end

Friday, May 16, 2014

Rails test with method that invoke delayed job

class Klass
def some_function
self.delay.time_consuming_function
end

def time_consuming_function
...
end
end

class KlassTest

test "some_function" do
Delayed::Job.all.each { |job| job.destroy }
Klass.new.some_function
Delayed::Job.all.each { |job| job.invoke_job }

assert ...
end

end

Monday, May 5, 2014

Postgresql server config files location

/etc/postgresql/[version]/main/

e.g. to set the listened port or location
/etc/postgresql/9.3/main/postgresql.conf

Tuesday, April 22, 2014

Single sign on for web applications

Trial on Atlassian Crowd

current application on
1. Ruby on rails - using devise gem
2. Java EE application.

to be cont....

Thursday, April 3, 2014

Postgresql kill process

select * from pg_stat_activity;
select pg_cancel_backend([procpid]);

What if you revoke an SSL cert?

After revoke, client may see on firefox (may not be the case on Chrome):


To by-pass the checking in firefox, do this but not recommended:


Uncheck the checking option under validation button.



Thursday, January 2, 2014

IT Director Responsibilities and Duties

IT Director Responsibilities and Duties


Coordinate with management in developing business strategic plans and operating policies.

Coordinate with business teams to analyze project contracts, proposals, requirements, and service level agreements.

Assist in developing project budgets and timelines.

Utilize project management experience to lead various global and regional projects.

Evaluate and revise project plan to meet changing project demands.

Manage project planning, staffing, schedule and expenses.

Oversee everyday project operations to ensure timely delivery.

Oversee the availability of application and infrastructure for project execution.

Identify and assign resources to IT projects.

Organize project status meetings with customers and team members.

Interact with customers to design and execute business solutions.

Ensure project deliverables meet highest quality standards and customer requirements.

Develop departmental standards and ensure that staffs adhere to these standards.

Provide analytical and technical assistance to project team.

Collaborate with management on application development, enhancement, and deployment activities.

Evaluate project proposals and change requests and make necessary recommendations for approval or rejection of proposals.