Thursday, December 23, 2010

Netbeans performance isues surfaced again

Last week, after I upgraded to the Netbeans 7.0 Beta, scanning issues, coupled with spontaneous freezes, cropped up again. I initially thought I was running into the same heap size issue as described in one of my earlier posts. After some research, it turns out the issue was related to garbage collection this time.

We've had several reports from users that choosing 'Concurrent Mark And Sweep' garbage collector improves scanning performance. This may or may not make a difference on your system. The problem is that people use different hardware, different versions of JDK and they have different default GC algorithm chosen by their JVM.

I added the required flags to my netbeans.conf file and performance improved to a point where it is now much better than it was previously.

netbeans_default_options="-J-client -J-Xss2m -J-Xms32m -J-Xmx3096m -J-XX:PermSize=32m -J-XX:MaxPermSize=1024m -J-Dapple.laf.useScreenMenuBar=true -J-Dsun.java2d.noddraw=true -J-XX:+UseConcMarkSweepGC -J-XX:+CMSClassUnloadingEnabled -J-XX:+CMSPermGenSweepingEnabled" 

My current configuration:

Intel(R) Xeon(R) Quad Core CPU @ 2.80GHz 
6G Memory
Multiple PHP projects containing a total of ~110,000 files.

java version "1.6.0_0"
OpenJDK Runtime Environment (IcedTea6 1.6) (fedora-23.b16.fc10-x86_64)
OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode)

Saturday, October 23, 2010

Using Google Nexus One in Istanbul Turkey

Yes, it is possible. The device is compatible and can connect to the local 3G network (Turkcell) with minimal hassle. In addition, even though I had to purchase a two-year T-mobile subscription with my phone, the device is unlocked!

Disclaimer: I purchased this phone directly from Google in May 2010. Google is no longer doing this.

As soon as I landed in Istanbul, I went to the Turkcell kiosk, registered my IMEI number, purchased a prepaid card, and activated it. You MUST register your phone with the government agency or it may be banned permanently.

The next step is to subscribe to the 3G internet service. There are two types of 3G services:

1. Faturali (invoiced) 3G Internet

Subscription for this plan is automatically renewed every month. And there is an overuse charge which is currently 0,050 TL/MB.

2. Faturasiz (non-invoiced) 3G Internet

This is an on-demand plan which is limited by time and/or usage. It does not renew automatically; you need to repurchase it at the end of your subscription period or if you hit your usage limit before your plan expires.

After getting a prepaid card, activating a 3G plan is as simple as sending an SMS message to the appropriate number defined on the links above. [In my case, I was not properly briefed by the sales people at the airport kiok so I ended up purchasing the first plan.]

The last step is to do a little bit of setup. Go to Settings > Mobile Networks > Access Point Names, click on Turkcell, and set the APN to "internet" - without quotes.

You should be able to connect to the Edge network. If not, reboot your phone. Some time later (for me it took over a day) you should receive an SMS confirmation message for your 3G subscription request and the 3G icon should replace the Edge icon. (There is an unnecessary opt-in process after you purchase a 3G package. It should happen automatically. If not, simply call the support center.)

Sunday, September 5, 2010

Removing the .php extension from Symfony controller

If you wish to shorten your Symfony 1.4 URLs ( by removing the php extension from the controller (, then read on.

1. Update your factories.yml file and add the relative_url_root option as described below:

    class: sfWebRequest
      logging:           %SF_LOGGING_ENABLED%
      path_info_array:   SERVER
      path_info_key:     PATH_INFO
      relative_url_root: ""
        txt:  text/plain
        js:   [application/javascript, application/x-javascript, text/javascript]
        css:  text/css
        json: [application/json, application/x-json]
        xml:  [text/xml, application/xml, application/x-xml]
        rdf:  application/rdf+xml
        atom: application/atom+xml

2. Create a symbolic link in your web directory to your controller:

cd /your/symfony/project/dir/web
ln -s symfonyApp.php symfonyApp

3. Update your apache configuration to set your new default type:

DefaultType application/x-httpd-php

Warning: This means all your text files would be passed through the PHP interpreter! There would be a performance hit depending on the number of static files that your server hosts.

4. Reload Apache configuration

service http reload

You should now be able to make successful requests to your syfmony application by using the shorter URL version.

Wednesday, August 25, 2010

Netbeans scanning issue resolved

I was having problems with Netbeans 6.9 where it would just hang trying to scan my codebase which consists of around ~200.000 PHP files. Another issue was related to code completion and suggestions which caused the IDE to become unresponsive or, in some cases, hang.

It looks like the issue is related to the heap size which is automatically set since version 6.0.

Since NetBeans version 6.0, the default limit for heap size (-J-Xmx) is determined automatically, with respect to the amount of memory available on the system. There is no -J-Xmx option specified in netbeans.conf. However, if you specify the heap size limit (i.e. you add the -J-Xmx... option to netbeans.conf), then the limit given by you will be respected.

So I ended up increasing the heap size in netbeans.conf by adding the "-J-Xmx2048m" option. Here are my current settings:

netbeans_default_options="-J-client -J-Xss2m -J-Xms32m -J-Xmx2048m -J-XX:PermSize=32m -J-XX:MaxPermSize=1024m -J-Dapple.laf.useScreenMenuBar=true -J-Dsun.java2d.noddraw=true" 

Although code completion is sometimes slow, my IDE never hangs and completes initial scanning within 2-3 hours.

Tuesday, May 18, 2010

Switched to Git

Finally decided to port all my work in my subversion repostiry to Git... A couple of tips that may be helpful for new starters:

1. In order to do an svn export, simply execute:

git archive | tar -x -C /path/to/dir

2. To revert local edits, run:

git checkout filename

3. The concept of remotes was confusing at the beginning. Think of them as shortcuts to remote repositories. So instead of typing complex URL's, all you need to do is

git push remote branch
git pull remote branch

once a remote is added.

3. Enjoy your merges :)

Sunday, May 16, 2010

Securing Postfix and Dovecot with TLS


This seemed to be confusing at first but here is what it boils down to:

With STARTTLS, an existing TCP connection is upgraded to an encrypted one after the SMTP handshake. On the other hand, with SSL/TLS, an ecnrypted connection is negiotiated right away before an SMTP handshake takes place. In other words, STARTTLS is "TLS inside SMTP", while SSL/TLS is "SMTP inside TLS".

See this page for more information.

Another important difference between these two schemes is that STARTTLS does not require a separate port. You can continue to use the same smtp (25) or imap (143) port. SSL/TLS on the other hand requires separate smtp (465) and imap (993) ports.


I wanted to implement a STARTTLS scheme; however, I decided to revert back to SSL/TLS due to:

1. I am running Dovecot dovecot-1.0.7 on CentOS release 5.5. Unfortunately for me, I was not able to require SSL connections since the "ssl = required" configuration option is not available until v1.2+. WIthout this I could not force TLS for non-plaintext authentication.

2. Outlook related issues described here.


Securing Postfix
The "smtpd_tls_wrappermode=yes" argument disables STARTTLS and enables SSL/TLS. It basically overrides the "smtpd_tls_security_level" flag inside /etc/postfix/ One thing to remember is that, you are not supposed to put this flag inside; it needs to be inside master.conf.
smtp      inet  n       -       n       -       -       smtpd
smtps     inet  n       -       n       -       -       smtpd -o smtpd_tls_wrappermode=yes 
Since we are using the smtps service, we need to punch a hole in our firewall for port 465.
smtpd_tls_cert_file = /etc/pki/tls/certs/mail.crt
smtpd_tls_key_file = /etc/pki/tls/private/mail.key
smtpd_tls_CAfile = /etc/pki/tls/certs/ca.crt
smtpd_tls_loglevel = 1
smtpd_tls_received_header = no
smtpd_tls_security_level = may
smtpd_tls_auth_only = yes
smtpd_tls_session_cache_timeout = 3600s
openssl s_client -tls1 -crlf -connect
Securing Dovecot
protocols = imaps # we need to open port 993 for this

disable_plaintext_auth = yes # Allows plaintext authentication only when SSL/TLS is used first.
ssl = required # v1.2+ only. Requires SSL/TLS also for non-plaintext authentication. 

ssl_cert_file = /etc/pki/tls/certs/mail.crt
ssl_key_file = /etc/pki/tls/private/mail.key
Testing Dovecot setup:
openssl s_client -tls1 -crlf -connect


Securing Postfix
smtp      inet  n       -       n       -       -       smtpd
#submission inet n       -       n       -       -       smtpd
As described previously, we can use an existing port with STARTTLS. Since we are using the usual smtp service, we need to punch a hole in our firewall for port 25. One other option is to use the submission service on port 587 to bypass ISP blocks.
smtpd_tls_cert_file = /etc/pki/tls/certs/mail.crt
smtpd_tls_key_file = /etc/pki/tls/private/mail.key
smtpd_tls_CAfile = /etc/pki/tls/certs/ca.crt
smtpd_tls_loglevel = 1
smtpd_tls_received_header = no
smtpd_tls_security_level = encrypt # This setting requires STARTTLS
smtpd_tls_auth_only = yes
smtpd_tls_session_cache_timeout = 3600s
If you are using a Postfix version older than v2.3, see smtpd_enforce_tls flag.

openssl s_client -starttls smtp -crlf -connect
Securing Dovecot
protocols = imap # No need for a separate port. We will stick with port 143.

disable_plaintext_auth = yes # Allows plaintext authentication only when SSL/TLS is used first.
ssl = required # v1.2+ only. Requires SSL/TLS also for non-plaintext authentication. 

ssl_cert_file = /etc/pki/tls/certs/mail.crt
ssl_key_file = /etc/pki/tls/private/mail.key
Testing Dovecot setup:
openssl s_client -starttls imap -crlf -connect

Saturday, March 27, 2010

Fedora 12 + Nagios + PNP4Nagios

Setting up pnp4nagios on Fedora is pretty straightforward.

1. Install pnp4 nagios
yum install pnp4nagios

2. Setup /etc/nagios/nagios.cfg

3. Setup /etc/nagios/objects/commands.cfg to send performance data to pnp4nagios.
define command{
        command_name    process-host-perfdata
        command_line    /usr/bin/perl /usr/libexec/pnp4nagios/ -d HOSTPERFDATA

define command{
        command_name    process-service-perfdata
        command_line    /usr/bin/perl /usr/libexec/pnp4nagios/

4. Setup: /etc/nagios/objects/yourserver.cfg
define host {
   name       host-pnp
   action_url /nagios/pnp4nagios/index.php?host=$HOSTNAME$&srv=_HOST_
   register   0

define service {
   name       srv-pnp
   action_url /nagios/pnp4nagios/index.php?host=$HOSTNAME$&srv=$SERVICEDESC$
   register   0

# a service using pnp
define service {
        use generic-service,srv-pnp
        hostgroup_name generic-hosts
        service_description Memory
        check_command check_nrpe!check_mem

5. Restart nagios

Official docs here.

Monday, February 15, 2010

MogileFS with Postgres

First of all, you need Postgres version 8.2 or newer since mogdbsetup refuses to run with older versions.

Anyways, I spent a quite a bit of time trying to install MogileFS with Postgres. I was getting "Can't create temporary test database:" errors. The answer was in the ~/.cpan/build/mogilefs-server-2.34/blib/lib/MogileFS/ file. The temp_store subroutine was defaulting to MySQL...

Here is the setup step by step on my Fedora 12:

yum install mogilefsd
yum install mogstored
yum install perl-CPAN // if needed
yum install perl-MogileFS-Client perl-MogileFS-Utils
yum install perl-DBD-Pg
yum install perl-IO-AIO

Setup services.

chkconfig --levels 345 mogilefsd on
chkconfig --levels 345 mogstored on

Setup a test database to be used during compilation.

$ createuser -SRlD mogile
$ createdb -E UTF8 -O mogile tmp_mogiletest

Setup environment variables for testing

$ MOGTEST_DBNAME=tmp_mogiletest

Download and install MogileFS::Store::Postgres from CPAN. (This is going to download the whole mogilefs-server package)

$ cpan
cpan> install MogileFS::Store::Postgres

Setup database

template1=# create user mogilefs with encrypted password 'password';
template1=# create database mogilefs with owner=mogilefs encoding='UNICODE';

Run mogdbsetup

mogdbsetup --type=Postgres --dbhost=localhost --dbname=mogilefs --dbuser=mogilefs --dbpass=password

Setup config

db_dsn DBI:Pg:dbname=mogilefs;host=mogilefs-db
db_user mogilefs
db_pass password
conf_port 6001
listener_jobs 5

Start tracker
service mogilefsd start

Update mogstored.conf

Define hosts and devices

mogadm host add mogilestorage-1 --ip= --port=7500 --status alive
mogadm host list
mogadm device add mogilestorage-1 1
mogadm device add mogilestorage-1 2
mkdir /var/mogdata/dev1
mkdir /var/mogdata/dev2
chown mogstored:mogstored /var/mogdata/dev1/
chown mogstored:mogstored /var/mogdata/dev2/
service mogstored start

Define domains and classes

mogadm domain add domain1
mogadm class add domain1 class1 --mindevcount=2

And don't forget to clean up the test user and database.

Sunday, February 14, 2010

Fedora 12 upgrade woes: Apache issues

Could not start the httpd service for a while due to:

Syntax error on line 196 of /etc/httpd/conf/httpd.conf: Cannot load /etc/httpd/modules/ into server: /etc/httpd/modules/ cannot open shared object file: No such file or directory

I had to manually comment out two unneeded modules from the /etc/httpd/conf/httpd.conf:

#LoadModule file_cache_module modules/
#LoadModule mem_cache_module modules/

More here.

Thursday, February 11, 2010

Passive checks with Nagios

Today, while I was trying to set up Nagios on my local box at work when Nagios refused to start due to bad configuration error.

Starting nagios:CONFIG ERROR! Start aborted. Check your Nagios configuration.

It took some to time to realize that a "check_command" line has to be defined even for passive check.

define service {
    use                        local-service
    service_description        passive_service
    check_command              check_dummy!2
    active_checks_enabled      0
    passive_checks_enabled     1

The best explanation is from the 2.0 docs located at:

What Happens When A Service Check Result Becomes "Stale"

If the check results of a service are found to be "stale" (as described above), Nagios will force an active check of the service by executing the command specified by the check_command option in the service definition. It is important to note that an active service check which is being forced because the service was detected as being "stale" gets executed even if active service checks are disabled on a program-wide or service-specific basis.

In the above example, if the service becomes "stale", the check_dummy command would simply reply with a critical(2) return code.

Saturday, February 6, 2010

Fedora 10 (Cambridge) to Fedora 12 (Constantine) with PreUpgrade

Ok, here goes my first blog post ever...

Last Friday, I decided to upgrade from Fedora 10 to 12 using the PreUpgrade tool. The download and the installation process went well until the time came to boot into the new system. I suspected something was wrong when I realized the fc12 kernel line was missing from the Grub menu, and I was "relieved" to see I was right the whole time when I was presented with:

Give root password for maintenance (or type Control-D to continue):

I could not even type. Every time I hit a key, i was prompted with the same line... Thankfully, I had that old custom compiled vanilla kernel sitting in the Grub menu that allowed me to boot and utilize the command line at least.

Although I was able to interact with Grub during the boot process, I decided to complicate the problem for no apparent reason and thought there must be something wrong with the grub itself... I burned the Fedora 12 DVD iso image and booted the computer into rescue mode... Two commands:

chroot /mnt/sysimage
grub-install /dev/sda

and a reboot later, I was staring at the same screen again...

This time I decided to check the /boot partition to see if the new kernel was properly installed. And voila: No fc12 vmlinuz or initrd files there! As usual, without any respect to conventional wisdom, I forcefully installed the first kernel rpm package that I could find on the Fedora 12 DVD. A quick check revealed that the required image files were created in the /boot partition along with the Grub entry. Confident, I rebooted with a smirk on my face only to discover a corrupted display - imagine a GUI with heavy texture and kaleidoscope effects.

A quick examination of the log revealed the following error:

Unable to load the kernel module 'nvidia.ko'

So I tried to reinstall the NVIDIA drivers:

yum remove kmod-nvidia
yum install kmod-nvidia

I had no luck there. Further investigation into /etc/rc.d/init.d/nvidia file clarified the loading process for nvidida.ko:

modpath="/lib/modules/$(uname -r)"
# Default to no module
# If one exists, then use it.
if test -e "${modpath}/extra/${modname}";then
elif test -e "${modpath}/extra/nvidia/${modname}";then
elif test -e "${modpath}/kernel/drivers/video/nvidia/${modname}";then

In my case, the older version of the kernel rpm package that I had forcefully installed was causing "uname -r" to return "". On the other hand, yum was installing everything into "/lib/modules/" due to the original kernel package that was also present on the system. I simply removed the old kernel package, reinstalled the original kernel package to generate the /boot images and the grub entry, and ran the nvidia install commands above...

After wasting six hours due to another logic exception, I got my computer working again.