Wednesday, December 31, 2014
java send mail using ubuntu local sendmail smtp
import javax.mail.Message;
import javax.mail.MessagingException;
import javax.mail.Session;
import javax.mail.Transport;
import javax.mail.internet.AddressException;
import javax.mail.internet.InternetAddress;
import javax.mail.internet.MimeMessage;
public static void main(String[] args) {
String to="recipient@example.com";
String from="sender@example.com";
Properties props = new Properties();
Session session = Session.getDefaultInstance(props, null);
// props.put("mail.smtp.host", "localhost"); // optional
String msgBody = "Sending email using JavaMail API...";
try {
Message msg = new MimeMessage(session);
msg.setFrom(new InternetAddress(from, "NoReply"));
msg.addRecipient(Message.RecipientType.TO,
new InternetAddress(to, "Mr. Recipient"));
msg.setSubject("Welcome To Java Mail API");
msg.setText(msgBody);
Transport.send(msg);
System.out.println("Email sent successfully...");
} catch (AddressException e) {
throw new RuntimeException(e);
} catch (MessagingException e) {
throw new RuntimeException(e);
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
}
Wednesday, December 24, 2014
Find largest directories on Linux
Thursday, December 18, 2014
Setup Zabbix on Ubuntu for Postgresql connection
2. Install postgresql odbc driver >sudo apt-get install postgresql-odbc
3. Setup /etc/odbc.ini
[test]
Description = 192.168.8.220 test
Driver = postgresql
Servername = 192.168.8.220
Port = 5432
Database = db_name
4. Setup /etc/odbcinst.ini
[postgresql]
Description = ODBC for postgresql
Driver = /usr/lib/x86_64-linux-gnu/odbc/psqlodbca.so
Setup = /usr/lib/x86_64-linux-gnu/odbc/libodbcpsqlS.so
Test:
>isql test your_name your_password -v
In Zabbix, add database monitor:
key = db.odbc.select[some_name,test]
Postgresql usage notes
REVOKE ALL ON DATABASE db_name FROM user_name;
revoke all on all tables in schema public from zabbix;
Readonly User:
revoke all on schema public from public;
create role zabbix login password 'some_pass';
grant usage on schema public to zabbix;
grant select on public.some_table to zabbix;
When you create a new database, any role is allowed to create objects in the public
schema. To remove this possibility, you may issue immediately after the database creation:
REVOKE ALL ON schema public FROM public;
Edit: after the above command, only a superuser may create new objects inside the public
schema, which is not practical. Assuming a non-superuser foo_user
should be granted this privilege, this should be done with:
GRANT ALL ON schema public TO foo_user;
Friday, December 12, 2014
Configure Zabbix to sent alert email
Setup Media Type, in this case, use email and local "sendmail" smtp server. Remember to use 127.0.0.1 on the SMTP server settings. Otherwise, it may have connection problem.
Ensure the User media email is enabled.
Ensure action is enabled.
Set the trigger.
Thursday, December 11, 2014
Connects zabbix server to agent
right server IP and other settings.
log: /var/log/zabbix-agent/zabbix_agentd.log
config: /etc/zabbix/zabbix_agentd.conf
ssh-copy-id copy id to remote machine
Tuesday, December 9, 2014
Monday, November 24, 2014
Wednesday, November 19, 2014
Jasperreport - the BI visualization tools
https://community.jaspersoft.com/ - community edition
Thursday, November 13, 2014
Business Intelligence
http://www.tableausoftware.com/products/desktop
http://www.qlik.com/
http://www.predictiveanalyticstoday.com/open-source-free-business-intelligence-solutions/
http://www.pentaho.com/determine-customer-value
https://www.g2crowd.com/categories/business-intelligence
https://www.jaspersoft.com/reporting-and-analytics-inside-your-application
http://www.slideshare.net/boyarski/the-5-levels-of-embedded-bi
Wednesday, November 12, 2014
Monitor linux process memory or cpu resource usage
>top -p pid
interactive commands:
"f" - add/drop columns
"c" - command details
"M" - sort by memory
...
Thursday, November 6, 2014
Talend built jar review
Trial run on SpagoBI all-in-one server
- Not stable.
- Used > 1GB RAM
- Require lots of resources to run.
Wednesday, November 5, 2014
Tools: SAP Crystal Reports
SAP Crystal Reports
A robust production reporting tool, SAP Crystal Reports turns almost any data source into interactive, actionable information that can be accessed offline or online, from applications, portals and mobile devices.
Friday, August 29, 2014
Postgresql JDBC enum and stored procedure
CallableStatement cs = conn.prepareCall("{call
sp_dummy(?::enum_result)}");
cs.setString(1, jobResult.toString());
cs.execute();
Thursday, August 28, 2014
Maven simple java application
Saturday, August 23, 2014
Work Breakdown Structure
Estimating using Work Breakdown Structure
- Setting up Development, Test, and Production environments - 8 hours
- Creating the Login screen - 4 hours
- Creating the Landing Page - 2 days
- Developing the XYZ Page - 12 hours
- etc., etc., etc.
WBS accuracy
Use feedback to improve WBS estimates
Don't forget acceptance testing
WBS estimate
Thursday, August 21, 2014
Jersey Maven setup Log4j2
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.0.1</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.0.1</version>
</dependency>
2. Add log4j configuration file to path: src/main/webapp/WEB-INF/classes
(create the classes folder manually)
3. Code:
Logger logger = LogManager.getLogger();
logger.info("info!!!!!!!!!!!!!!");
Serialize Object to Json indented string with Jersey
<dependency>
<groupId>org.glassfish.jersey.media</groupId>
<artifactId>jersey-media-json-jackson</artifactId>
</dependency>
2. Code:
ObjectMapper mapper = new ObjectMapper();
mapper.enable(SerializationFeature.INDENT_OUTPUT);
String rJson = "";
try {
rJson = mapper.writeValueAsString(request);
System.out.println(rJson);
} catch (JsonProcessingException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
Wednesday, August 20, 2014
Tomcat 7 connection pool with JNDI datasource and postgreSQL
<Context> <Resource name="jdbc/postgres" auth="Container" type="javax.sql.DataSource" driverClassName="org.postgresql.Driver" url="jdbc:postgresql://127.0.0.1:5432/mydb" username="myuser" password="mypasswd" maxActive="20" maxIdle="10" maxWait="-1"/> </Context>
2. Web application's web.xml
<resource-ref> <description>postgreSQL Datasource example</description> <res-ref-name>jdbc/postgres</res-ref-name> <res-type>javax.sql.DataSource</res-type> <res-auth>Container</res-auth> </resource-ref>
2. Code
Connection conn = null;
try {
InitialContext cxt = new InitialContext();
DataSource ds = (DataSource) cxt.lookup( "java:/comp/env/jdbc/postgres" );
conn = ds.getConnection();
...
...
while(rs.next()) {
...
...
}
rs.close();
}
conn.close();
} catch (NamingException e1) {
e1.printStackTrace();
} catch (SQLException e) {
e.printStackTrace();
};
Maven Postgresql JDBC
http://mvnrepository.com/artifact/org.postgresql/postgresql/9.3-1100-jdbc41
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>9.3-1100-jdbc41</version>
</dependency>
2. Coding:
Connection conn = null;
try
{
Class.forName("org.postgresql.Driver");
String url = "jdbc:postgresql://THE_HOST/THE_DATABASE";
conn = DriverManager.getConnection(url,"THE_USER", "THE_PASSWORD");
Statement st = conn.createStatement();
ResultSet rs = st.executeQuery("SELECT id, subject, permalink
FROM blogs ORDER BY id");
while ( rs.next() )
{
Blog blog = new Blog();
blog.id = rs.getInt ("id");
blog.subject = rs.getString ("subject");
blog.permalink = rs.getString ("permalink");
listOfBlogs.add(blog);
}
rs.close();
st.close();
}
catch (ClassNotFoundException e)
{
e.printStackTrace();
System.exit(1);
}
catch (SQLException e)
{
e.printStackTrace();
System.exit(2);
}
Monday, August 18, 2014
System server requirements
System hardware requirements | ||||
CPU | File I/O | Database I/O | RAM | |
Management Web App | High | Low | High | Mid |
Scoring Web App | Low | Low to Mid | Mid | Mid |
Workers | High | High | Mid | High |
PostgreSQL | Mid | High | N/A | High |
DWH | N/A | N/A | High | N/A |
ETL | High | Low to Mid | Mid | High |
CRM | Mid | Low | Mid | Mid |
Analytics Engine | High | Low | Mid | High |
System Server Requirements | ||||
File Server | Database Server | Web Server | Processing Server | |
Functionalities | SFTP, NFS | DWH, Operation DB | Web based application, CRM, … | Workers, ETL, Analysis Engine, … |
CPU | Low (E3-1230v2 or existing) | Mid to High (E5-2420 dual / E52620 dual) | Mid (E3-1230v2/existing server) | High (E3-1230v2/existing server) |
RAM | 16G or above | 32G or above | 16G or above | 16G or above |
RAID | RAID 1/5/6 | RAID 10 | NO RAID / RAID 1 | NO RAID |
RAID Controller | Dell H710p or equivalent | Dell H710p or equivalent | N/A | N/A |
HDD | TBD | 1TB x 4 | 500GB | 500GB |
Quantity | x1 | x1 | xN | xN |
Public IPs | x1 | N/A | x2 | N/A |
Region | All | All | All | All |
Scalability | Add drive for increase storage capacity. Can add new server as required. | Single server can support up to 3 to 5 years. Can add new server as required. | Scale Horizontally by adding new server as required. | Scale Horizontally by adding new server as required. |
New Server Price | 20K (Dell R220) to 40K (Dell R520) | 50K+ HKD (Dell R520/R720) | 12K HKD (Dell R220) | 12K HKD (Dell R220) |
Servers Allocation | ||||
Live site | ||||
File Server | Database Server | Web Server | Processing Server | |
New server Dell R220/R520 | New server Dell R520/R720 | HK | ||
20K/40K+ HKD - Stage 2+ | 50K+ HKD - Stage 2 | existing server #1 - Management | ||
Support both all regions | existing server #2 - Scoring | |||
Database for Management, Scoring, Worker, DWH, etc. | Dell R210 - Workers, ETL, SFTP | |||
UK | ||||
existing #3 - Management | ||||
existing #4 - Scoring | ||||
=R220, R210 #2 in Stage 1 | = existing servers, R210 and R220 servers in Stage 1 | Dell R220 #2 (12K HKD - Stage 1) - Worker, SFTP | =R210, R220 #2 in Stage 1 | |
Standby site | ||||
File Server | Database Server | Web Server | Processing Server | |
New server Dell R220 RAID 1 | New server Dell R220 with RAID 1 | HK | ||
~20K HKD - Stage 3 | (* Use low to mid-range standby server) | R220 #3 (12K HKD - Stage 1.5) - Management, Scoring, SFTP, … | ||
~20K HKD - Stage 3 | 1 x VM #1 (2K HKD - Stage 1.5) Worker | |||
UK | ||||
R220 #3 (12K HKD - Stage 1.5) - Management, Scoring, SFTP, … | ||||
=R220 #3, R220 #4, VM #1, VM #2 in Stage 1.5 | =R220 #3, R220 #4, VM #1, VM #2 in Stage 1.5 | 1 x VM #2 (2K HKD - Stage 1.5) Worker | =VM #1, VM #2 in Stage 1.5 |
Thursday, August 14, 2014
Linux, Ubuntu log files location
log message to this file
=> /var/log/dmesg : Linux kernel ring buffer log
=> /var/log/dpkg.log : All binary package log includes package
installation and other information
=> /var/log/faillog : User failed login log file
=> /var/log/kern.log : Kernel log file
=> /var/log/lpr.log : Printer log file
=> /var/log/mail.* : All mail server message log files
=> /var/log/mysql.* : MySQL server log file
=> /var/log/user.log : All userlevel logs
=> /var/log/xorg.0.log : X.org log file
=> /var/log/apache2/* : Apache web server log files directory
=> /var/log/lighttpd/* : Lighttpd web server log files directory
=> /var/log/fsck/* : fsck command log
=> /var/log/apport.log : Application crash report / log file
=> /var/log/messages : General log messages
=> /var/log/boot : System boot log
=> /var/log/debug : Debugging log messages
=> /var/log/auth.log : User login and authentication logs
Tuesday, August 12, 2014
Using pg_prove with pgTap
#> create extension pgtap;
>\q
>pg_prove -d my_database some_test_file.sql
install pg_prove for pgtap
>sudo ./Build
>sudo ./Build installdeps
>sudo ./Build test
>sudo ./Build install
DONE
Check Ubuntu ethernet interface speed.
Wednesday, August 6, 2014
Measure the bandwidth between 2 servers
> sudo apt-get install iperf
Computer A:
> iperf -s
Computer B:
> iperf -c address_of_computer_A
Ubuntu Server reconfigure timezone
>more /etc/timezone
>sudo dpkg-reconfigure tzdata
>sudo service cron stop
>sudo service cron start
Wednesday, July 9, 2014
Google Map simple marker
<html>
<head>
<meta name="viewport" content="initial-scale=1.0, user-scalable=no">
<meta charset="utf-8">
<title>Geocoding service</title>
<style>
html, body, #map-canvas {
height: 100%;
margin: 0px;
padding: 0px
}
#panel {
position: absolute;
top: 5px;
left: 50%;
margin-left: -180px;
z-index: 5;
background-color: #fff;
padding: 5px;
border: 1px solid #999;
}
</style>
<script src="https://maps.googleapis.com/maps/api/js?v=3.exp"></script>
<script
src="http://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
<script>
var geocoder;
var map;
function initialize() {
geocoder = new google.maps.Geocoder();
var latlng = new google.maps.LatLng(-34.397, 150.644);
var mapOptions = {
zoom: 8,
center: latlng
}
map = new google.maps.Map(document.getElementById('map-canvas'),
mapOptions);
}
var address_array =
[
"Shop 3-5, G/F, Po Fung Building, 1-7 Hop Yick Road, Yuen Long ",
"Rm 401, 4/F, Sincere House, 83 Argyle Street, Mong Kok ",
"Shop M001, Long Ping Commercial Complex, Long Ping Estate, Yuen Long ",
"Shop G06, G/F, Spot, 48 Lung Sum Avenue, Shek Wu Hui, Sheung Shui ",
"Unit 3, 2/F, United Success Commercial Centre, 508 Jaffe Road, Causeway
Bay "
];
count = 0
function codeAddress() {
//var address = document.getElementById('address').value;
var address = address_array[count];
if(typeof(address) == 'undefined') {
alert("Map completed.");
clearInterval(intervalID);
return;
}
geocoder.geocode( { 'address': address}, function(results, status) {
if (status == google.maps.GeocoderStatus.OK) {
map.setCenter(results[0].geometry.location);
var marker = new google.maps.Marker({
map: map,
position: results[0].geometry.location
});
} else {
if (status == google.maps.GeocoderStatus.OVER_QUERY_LIMIT) {
//alert("OVER_QUERY_LIMIT, just press ok to continue :) ");
count--;
} else {
the_index = address.indexOf(',');
if(the_index > 0) {
address_array.push(address.substring(the_index + 1,
address.length));
}
//alert('Geocode was not successful for the following reason: '
+ status);
//var error_messages = $('#messages').html() + status + " --- "
+ address + "<br/>"
//$('#messages').html(error_messages);
}
}
});
count++;
}
google.maps.event.addDomListener(window, 'load', initialize);
$(document).ready(function() {
//var intervalID = window.setInterval(codeAddress(), 1000);
start_map();
});
</script>
<script>
var intervalID;
function start_map() {
intervalID = window.setInterval(function(){codeAddress()}, 1000);
}
</script>
</head>
<body>
<div id="panel">
<!--
<input id="address" type="textbox" value="Sydney, NSW">
<input type="button" value="Geocode" onclick="start_map()">
-->
</div>
<div id="map-canvas" ></div>
<div id="messages" style='font-family:arial;' >The following
addresses cannot be marked: <br/></div>
</body>
</html>
Monday, June 23, 2014
Rails - Make Delayed job work with Rspec in tests
Delayed::Worker.delay_jobs = false
Friday, June 20, 2014
Rails override Time.now to return Time.zone.now
class << Time
alias :system_now :now
def now
ActiveSupport::TimeWithZone.new(self.system_now.utc, Time.zone)
end
end
Tuesday, June 17, 2014
Nagios create plugins
Create plugin in /usr/lib/nagios/plugins (in C/Perl/Script/etc)
Enable plugin by:
1. define comment in /etc/nagios-plugins/config/somefile.cfg
2. edit /etc/nagios/conf.d/localhost_nagios2.cfg
Restart:
sudo service nagios3 restart
Thursday, June 12, 2014
Reviewed Ubuntu Server monitoring tools
1. Nagios
2. Munin
3. Cacti
4. Zabbix
Turn out I like both Nagios and Zabbix.
Wednesday, June 11, 2014
Setup Nagios on Ubuntu 12.10
sudo apt-get install nagios3 nagios-nrpe-plugin
If you have other web pages or apps running on the same server, the following information might be useful to separate Nagios as it's own Apache virtual host on a non-standard port (for example 43326 here).
Remove standard config and create Apache virtual host;
sudo cp /etc/apache2/conf.d/nagios.conf /etc/apache2/sites-available/nagio s3 sudo rm /etc/apache2/conf.d/nagios.conf sudo vi /etc/apache2/sites-available/nagios3
To beginning of file add;
Listen 43326 <VirtualHost *:43326> ServerAdmin webmaster@localhost DocumentRoot /var/www/nagios3
To the end of the file add;
</VirtualHost>
Enable the new site;
sudo a2ensite nagios3
If you run a firewall (UFW), open the port;
sudo ufw allow 43326
Restart Apache
sudo service apache2 restart
Linux system monitoring
http://www.tecmint.com/command-line-tools-to-monitor-linux-performance/
http://www.cyberciti.biz/tips/top-linux-monitoring-tools.html
http://www.thegeekstuff.com/2011/12/linux-performance-monitoring-tools/
http://www.opensourceforu.com/2013/12/look-top-three-network-monitoring-tools/
Tuesday, June 3, 2014
Wednesday, May 21, 2014
Rails: Specify Time.zone in delayed job
The method_missing chain will add the current Time.zone.name to the
function's argument.
The Time.zone.name will be popped by the PerformableMethod.perform chain.
Delayed::DelayProxy.class_eval do
# Embed the current time zone name to delayed job's arguments when
# adding delayed job from delay() method.
def method_missing_with_custom_time_zone(method, *args)
raise "Time zone is nil" if Time.zone.nil?
args << {:custom_time_zone => Time.zone.name}
result = method_missing_without_custom_time_zone(method, *args)
result
end
alias_method_chain :method_missing, :custom_time_zone
end
Delayed::PerformableMethod.class_eval do
# Read time zone from job's argument.
def perform_with_custom_time_zone
if self.args.last.is_a?(Hash) &&
!self.args.last[:custom_time_zone].nil?
Time.zone = args.pop[:custom_time_zone]
else
raise "Custom time zone is missing from the arguments."
end
perform_without_custom_time_zone
end
alias_method_chain :perform, :custom_time_zone
end
Friday, May 16, 2014
Rails test with method that invoke delayed job
def some_function
self.delay.time_consuming_function
end
def time_consuming_function
...
end
end
class KlassTest
test "some_function" do
Delayed::Job.all.each { |job| job.destroy }
Klass.new.some_function
Delayed::Job.all.each { |job| job.invoke_job }
assert ...
end
end
Monday, May 5, 2014
Postgresql server config files location
e.g. to set the listened port or location
/etc/postgresql/9.3/main/postgresql.conf
Wednesday, April 23, 2014
Tuesday, April 22, 2014
Single sign on for web applications
current application on
1. Ruby on rails - using devise gem
2. Java EE application.
to be cont....
Thursday, April 3, 2014
Postgresql kill process
select pg_cancel_backend([procpid]);
What if you revoke an SSL cert?
To by-pass the checking in firefox, do this but not recommended:
Uncheck the checking option under validation button.
Thursday, March 6, 2014
Tuesday, February 11, 2014
Ubuntu boot startup time log graph
Thursday, February 6, 2014
Check installed software on windows
wmic:root\cli>/output:C:\InstallList.txt product get name, version
wmic:root\cli>exit
Monday, January 20, 2014
Monday, January 6, 2014
Ubuntu: Change the waiting time on network configuration
Thursday, January 2, 2014
IT Director Responsibilities and Duties
Coordinate with management in developing business strategic plans and operating policies.
Coordinate with business teams to analyze project contracts, proposals, requirements, and service level agreements.
Assist in developing project budgets and timelines.
Utilize project management experience to lead various global and regional projects.
Evaluate and revise project plan to meet changing project demands.
Manage project planning, staffing, schedule and expenses.
Oversee everyday project operations to ensure timely delivery.
Oversee the availability of application and infrastructure for project execution.
Identify and assign resources to IT projects.
Organize project status meetings with customers and team members.
Interact with customers to design and execute business solutions.
Ensure project deliverables meet highest quality standards and customer requirements.
Develop departmental standards and ensure that staffs adhere to these standards.
Provide analytical and technical assistance to project team.
Collaborate with management on application development, enhancement, and deployment activities.
Evaluate project proposals and change requests and make necessary recommendations for approval or rejection of proposals.