Install GeoNode for Development – The Working Version

Posted by

0

Install GeoNode for Development

In order to install Geonode 2.0 in developing mode on Ubuntu 12.04 the following steps are required:

  1. install build tools and libraries
  2. install dependencies (Python and Java) and supporting tools
  3. add PPA repository
  4. set up a virtual environment (virtualenv)
  5. clone geonode from github and install it in the virtual environment
  6. run paver to get install geoserver and start the development servers

Note

The following steps have to be executed in your terminal. The steps have to be done as a root user, therefore don´t forget to type sudo in front!

  1. retrieve latest apt-get list

    $ sudo apt-get update

  2. Install build tools and libraries

    $ sudo apt-get install -y build-essential libxml2-dev libxslt1-dev

  3. Install dependencies

    Python native dependencies

    $ sudo apt-get install -y python-dev python-imaging python-lxml python-pyproj python-shapely python-nose python-httplib2 python-pip python-software-properties

    Install Python Viturual Environment

    $ sudo pip install virtualenvwrapper

    Java dependencies

    $ sudo apt-get install -y --force-yes openjdk-6-jdk ant maven2 --no-install-recommends

    supporting tools

    $ sudo apt-get install -y git gettext

  4. Node and tools required for static development

    This is required for static development

    $ sudo add-apt-repository -y ppa:chris-lea/node.js
$ sudo apt-get update
$ sudo apt-get install -y nodejs
$ sudo npm install -y -g bower
$ sudo npm install -y -g grunt-cli

  5. Set up a virtual environment

    Here is where Geonode will later be running.

    Add the virtualenvwrapper to your new environement

    $ export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python
$ export WORKON_HOME=~/.venvs
$ source /usr/local/bin/virtualenvwrapper.sh
$ export PIP_DOWNLOAD_CACHE=$HOME/.pip-downloads
>> ADDED BY J2ON
$ export LD_LIBRARY_PATH=/usr/local/lib

    set up the local virtual environment for Geonode   [HERE IS THE PLACE YOU FEEL -_-a]

    $ mkvirtualenv geonode
$ workon geonode

    This creates a new directory where you want your project to be and creates a new virtualenvironment

  6. Get the code

    To download the latest geonode version from github, the command clone is used

    $ git clone https://github.com/GeoNode/geonode.git

  7. Install GeoNode in the new active local virtualenv

    $ pip install -e geonode --use-mirrors
$ cd geonode

If the install fails because of an error related to pyproj not being verified (happens on pip 1.5), use the following:

$ pip install -e geonode --use-mirrors --allow-external pyproj --allow-unverified pyproj

  8. Compile and Start the server

    The last step is to compile GeoServer and setup

    $ paver setup

    >> ADDED BY J2ON

    GeoNode development environment successfully set up.If you have not set up an administrative account, please do so now. Use “paver start” to start up the server.

  9. Now we can start our geonode instance

    $ paver start

    Visit the geonode site by typing http://localhost:8000 into your browser window.

  10. To stop the server

    type hold Ctrl c on your keyboard to stop the server

    now type:

    $ paver stop    # to stop all django, geoserver services

  11. Next create a superuser for your django geonode

    Create a superuser so you can log on to your local geonode installation athttp://localhost:8000

    $ django-admin.py createsuperuser –-settings=geonode.settings

Linear Regression

Posted by

0

Linear Regression Basic
§ Appropriate for con1nuous dependent variables
– Such as income, height, dura-on, age, speed, or temperature
§ Basic formula for linear regression: Y = α + Xβ + ε
– Y is the dependent variable
– X is an independent variable
– Beta is a coefficient that shows change in Y per change in X – Alpha is the value where the line intercepts the Y axis at X = 0 – Epsilon represents a random distribu-on of error

CDH and Cloudera Manager on EC2 instance with ephemeral drive – preface

Posted by

0

When you use d2 instance type on AWS, ephemeral drives may need to be reformatted and /etc/hosts files should be all formatted with mkfs.
I’m starting this blog as a series of EC2 challenge with CDH.

http://www.cloudera.com/resources/recordedwebinar/five-tips-for-running-cloudera-on-aws.html
Next time, ephemeral drive with lsblk, fdisk, mkfs, and mount….

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-using-volumes.html

 

 

 

No-Admin series #2 Web Developer’s option with Node.js

Posted by

0

When a server is available such as Ubuntu or CentOS to ssh in, you may not even look at this option. However, if you have limited environment with no-admin privilege, this is another article that you need for Web-dev environment setup.

  • Node.js
  • Npm
  • Bower
  • Gruntjs
  • etc…

http://abdelraoof.com/blog/2014/11/11/install-nodejs-without-admin-rights/

Now set up Node and Npm

How to install Node.js without admin rights

 Published On November 11, 2014

Node.js does provide pre-built installers for all platforms but here we are going to talk about how to install and prep your PC for Node.js development when you don’t have administrator rights.

Step 1: Get Node.exe

First you will need to download the Windows Binary. You can get it from Node.js download page or http://nodejs.org/dist/latest/. For x64, you will have to download from the appropriate folder. Move the executable to a local folder.

Step 2: Get NPM

NPM (Node Package Manager) is the package manager for Node.js and you will need this for your development. You can download NPM from http://nodejs.org/dist/npm/ and extract the zip file to a local folder.

Step 3: Configure the environment PATH variable.

You need to set up the PATH variable so that you can call node from anywhere in the system.

set PATH=%PATH%;D:\path-to-your-node;D:\path-to-your-npm

Mostly likely, the usual way of setting environment variables may not be accessible or not sufficient for you. instead you might want to hit Win + R (Open the Run dialog) and execute this:

rundll32 sysdm.cpl,EditEnvironmentVariables

This command will provide you with Environment Variables dialog box. You will be able to add/modify the PATH variable for the current user.Environment Variables dialog box

Step 4: Testing your Node.js installation

Quick way to test your Node.js installation is to get the version of Node.js that you are running by running the following command:

node --version

You should get something like this:Testing Nodejs installation

Extra: Setting up NPM to work behind a proxy

Apart from the hurdle of not having admin rights, one common issue is working behind the corporate proxy. Fire up your command line and type in the following:

npm config set proxy http://proxy:port
npm config set https-proxy https://proxy:port

If you need to specify the credentials, then use the following syntax:

npm config set proxy http://username:password@proxy:port
npm config set https-proxy https://username:password@proxy:port

Now you can get your hands dirty with Node.js. Happy programming! 🙂

No-Admin Series#1: JDK with no Admin on Windows (why do we even have this problem in IT?)

Posted by

0

First, it’s sad that we even have this issue with no-admin Windows environment at work places for us.

The world somehow thinks Developers don’t need Admin privilege on their workstations. Then, developers are supposed to learn and train themselves to industry standards and new technologies while they are finishing up the tasks that they are asked to do.

However, we are developers, who understand the infrastructure and programming, technically know we should be able to use some development tools and standards libraries such as JDK. So, here’s how in case you want on one place.

What you need:

Unpacking tool for JDK**.exe, JDK installation file that you can download from Oracle. I suggest installing Cygwin with no-admin option for future tools option if you don’t already have.

  1. Cygwin: (https://www.cygwin.com/). Download, and run setup**.exe with –no-admin option on Command Prompt.
  2. Choose base and other tools you want to use on bash as well as “cabextract”. Finish the installation.
  3. Open Cygwin window and find /cygdrive/c/{your download directory where JDK installation file is}
  4. Type on the prompt > cabextract jdk1.**.exe (where jdk1.**.exe is the installation file)
  5. Now that you have extracted file called tools.zip along with other files and folders.
  6. Unzip tools.zip and move the extracted folder “tools” to where you want JDK (JAVA_HOME) to be and rename as you wish.
  7. Remember to set environment variable for you for JAVA_HOME to the folder tools, or renamed folder of tools.
  8. From cygwin prompt, run “find . -name ‘*.pack‘” to find all the .pack files and run the following command for each pack file on Command prompt.
  9. C:\JDK_HOME> unpack200 {file_name}.pack {file_name}.jar
  10. (eg: .\jre\bin\unpack200 .\jre\lib\rt.pack .\jre\lib\rt.jar)
  11. Now, your JDK and JAVA_HOME should be set.
  12. You can run binary version of Eclipse to test or other program to use JAVA_HOME

TortoiseSVN svn+ssh problem with putty in VM Windows on Mac Host

Posted by

0

First, I spent two days of Googling about this problem. The results I found are all useless. That means people don’t use VM guest Windows and TortoiseSVN much. Most of them install Putty instead of using the binary version. Also, they have other issues more.

Anyway, just for the people out there who try to use TortoiseSVN on Windows VM (I don’t know why only on VM guest. It works fine on host machines), use TortoisePLink for Settings/Network/SSH/SSH client: “%Drive%\Program Files\TortoiseSVN\bin\TortoisePlink.exe”

I remember this worked fine with Putty.exe or Plink.exe. It still worked fine on another machine, not VM guest though.

The reason may be the version incompatibility.

Just for you to try another option!

Hadoop memory solution?

Posted by

0

http://stackoverflow.com/a/3384470/884569

Job failing or is your server crashing? If your Job is failing because of OutOfMemmory on nodes you can tweek your number of max maps and reducers and the JVM opts for each so that will never happen. mapred.child.java.opts (the default is 200Xmx) usually has to be increased based on your data nodes specific hardware.

http://allthingshadoop.com/2010/04/28/map-reduce-tips-tricks-your-first-real-cluster/

Max tasks can be setup on the Namenode or overridden (and set final) on data nodes that may have different hardware configurations. The max tasks are setup for both mappers and reducers. To calculate this it is based on CPU (cores) and the amount of RAM you have and also the JVM max you setup in mapred.child.java.opts (the default is 200). The Datanode and Tasktracker each are set to 1GB so for a 8GB machine the mapred.tasktracker.map.tasks.maximum could be set to 7 and the mapred.tasktracker.reduce.tasks.maximum set to 7 with the mapred.child.java.opts set to -400Xmx (assuming 8 cores). Please note these task maxes are as much done by your CPU if you only have 1 CPU with 1 core then it is time to get new hardware for your data node or set the mask tasks to 1. If you have 1 CPU with 4 cores then setting map to 3 and reduce to 3 would be good (saving 1 core for the daemon).

By default there is only one reducer and you need to configure mapred.reduce.tasks to be more than one. This value should be somewhere between .95 and 1.75 times the number of maximum tasks per node times the number of data nodes. So if you have 3 data nodes and it is setup max tasks of 7 then configure this between 25 and 36.

If your server is crashing with OutOfMemory issues then that is where the HADOOP_HEAPSIZE comes in just for the processes heap (not the execution of task).

Lastly, if your Job is taking that long you can check to see if you have another good configuration addition is mapred.compress.map.output. Setting this value to true should (balance between the time to compress vs transfer) speed up the reducers copy greatly especially when working with large data sets. Often jobs do take time but there are also options to tweak to help speed things up =8^)