Saturday, February 4, 2023

Remote connection to a Jupyter notebook

 This is explained here

https://docs.anaconda.com/anaconda/user-guide/tasks/remote-jupyter-notebook/


Follow the following steps to use Jupyter Notebook launched from remote server.


Launch Jupyter Notebook from remote server, selecting a port number for <PORT>:


# Replace <PORT> with your selected port number

jupyter notebook --no-browser --port=<PORT>

For example, if you want to use port number 8080, you would run the following:


jupyter notebook --no-browser --port=8080

Or run the following command to launch with default port:


jupyter notebook --no-browser

Please note the port setting. You will need it in the next step.


You can access the notebook from your remote machine over SSH by setting up a SSH tunnel. Run the following command from your local machine:


# Replace <PORT> with the port number you selected in the above step

# Replace <REMOTE_USER> with the remote server username

# Replace <REMOTE_HOST> with your remote server address

ssh -L 8080:localhost:<PORT> <REMOTE_USER>@<REMOTE_HOST>

The above command opens up a new SSH session in the terminal.


Open a browser from your local machine and navigate to http://localhost:8080/, the Jupyter Notebook web interface. Replace 8080 with your port number used in step 1.


---------------------------------------------------

If you want to use jupyter lab instead of notebook and connect remotely from visual studio you can follow these steps.


1. First start a jupyter lab server on the remote server:

jupyter lab --no-browser --NotebookApp.allow_origin='*' --port=8080


2. Now from visual studio connect to this server using the Jupyter extension

https://stackoverflow.com/questions/72121390/how-to-use-jupyterlab-in-visual-studio-code

you might have to install the extension first.

Tuesday, March 10, 2020

Monday, April 3, 2017

Language Modelling Datasets and Tools

An introduction to language modelling (LM) is here
For RNN LMs see Mikolov's slides here
Mikolov's RNN LM tool kit  is here
SRI international's language modelling tool kit is here

Language modelling can be slow with RNNs. This is a faster implementation that uses Eigen.

There is a 1B token benchmark dataset released by Google for evaluating language models. It can be obtained here It has tokenised and splitted heldout and train portions. Shard 0000 is used for reporting test results.


Thursday, May 5, 2016

MongoDB cheat sheet

To run mongodb do
>mongod
This will start the deamon.

Then to query the DB start the client by
>mongo
 To use a DB do
use

To see the databases do
show databases

You can see the collections in a database after opening that database by
show collections

Collections correspond to Tables in mysql.

db.collection.find({key:value})

can be used to query the DB. If you want to perform "like" queries do
db.collection.find({"my key": /term/})

Do not use quotes around term in the previous example to perform like queries. If you do, then it will be an exact match.

This will not work within pymongo and you need to use $regex directive for this.
db["mycollection"].find({"key":{"$regex":"%s" % query}})

You can use count do count the number of matching records.
db.collection.count({key:value})



Friday, March 4, 2016

Bibdesk annotations in TeX preview

If you would like to display the annotations you do in bibdesk in the TeX preview then you can do so as follows.

The annotations you make are kept in a bibtex field called "Annote".

We need a bibtex style file (bst) that supports this annote field.
You can copy paste the plainannotate.bst file at the end of this post.

Next, create the following directory
~/Library/texmf/bibtex/bst
and put the style file there.

Now from Bibdesk set the preferences as follows


That is it



Tuesday, March 24, 2015

texshop 日本語入力

texshopで背景色を黒にしている場合,日本語入力がまだ未確定の場合,入力している文字が見えなくなるという問題がある.これは良くするためには次のコマンドを使えば良い.

defaults write TeXShop ResetSourceTextColorEachTime YES

しかし,これでバグがでることもあるようなので注意.

参照:
http://pages.uoregon.edu/koch/texshop/version.html

Tuesday, January 6, 2015

Monday, January 5, 2015

Python tricks and tips

If you want to assign incremental ids to a list of words then you can define a defultdict as follows:

wpids = defaultdict(lambda: len(wpids))

Thursday, December 11, 2014

boost-python

Here is a minimum working example of boos-python from the official web site.

First create a hello.cpp file and put this.

#include YOU HAVE TO INCLUDE boost/python.hpp within tag markers.

char const* greet()
{
   return "hello, world";
}


BOOST_PYTHON_MODULE(hello)
{
    using namespace boost::python;
    def("greet", greet);
}

Now compline .so file by doing the following.

g++ -g -shared -fPIC -I/usr/include/python2.7 hello.cpp -lpython2.7 -lboost_python -o hello.so

Open IPython and import hello
now call hello.greet()


Monday, September 22, 2014

Distributed vs. Distributional Representations

Distributed Representations = Learnt using Deep Learning methods. Distributed over the parameter space. Dense, lower dimensional embeddings.

Distributional Representations = Counted from a corpus. Classical distributional hypothesis. Sparse, High dimensional.

Saturday, September 20, 2014

My referencing workflow

I use BibDesk for reference management. After a major academic event such as a conference, I access the proceedings website using BibDesk. ACL anthology indexes all papers for NLP conferences, IEEE Explorer can be used for IEEE conferences, ACM digital library works for ACM conferences, and Google Scholar in general for any. The BibTex entries can be imported to BibDesk from within BibDesk with a url to the paper. That is it. I then assign papers a To READ tag (keyword) or to a Prority Reading list. I then read those papers and assign keywords and annotations. I use Skim for reading and annotating PDFs on Mac.

For PDFs I find online, I send them to CiteULike and then import them to BibDesk from there. CiteULike can extract most of the attributes from PDFs (scientific papers) such as title and authors. I can use BibDesk again to access CiteULike and import these references.

Mendely is also a good tool if you want to find related papers.

This is my workflow.

Sunday, September 14, 2014

Remove duplicate entries from launch services

Run the following command as explained here

/System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Support/lsregister -kill -r -domain local -domain system -domain user

Thursday, July 31, 2014

Debugging Python using ipdb

We can install ipdb (interactive python debugger) via pip
$ pip install ipdb
To insert a breakpoint (hard coded break points) in the code use ipdb.set_trace(). You can insert many breakpoints as you like. To run the code:
$ python mycode.py
When the breakpoint is hit, you will be dropped to ipdb prompt, similar to the ipython prompt. Use "c" (continue) command to continue the code until the next breakpoint or until the program ends.

For example,


import sparsesvd
import ipdb

def f(x):
    print x
    y = 2 *x 
    ipdb.set_trace()
    print y
    z = y * 3
    ipdb.set_trace()
    print z
    pass

if __name__ == "__main__":
    f(10)
    print "all done"

Wednesday, July 30, 2014

Bitbucket with SSH

If you want to access bitbucket via SSH instead of https, upload the SSH RSA public key to bitbucket. Then use the following path for example when cloning.

git clone ssh://git@bitbucket.org/Bollegala/SupEmb.git

Thursday, June 19, 2014

pbibtex Japanese processing

If you are using pbibtex in TexShop and have Japanese bib file encoded in sjis, then you need to specify the kanji code as sjis in order to properly format the Japanese bibtext entries. This can be done either by command line as
$ pbibtex projname -kanji=sjis
or by the TexShop Preferences --> Engine dialog under default bibtex engine specification by setting
pbibtex -kanji=sjis

Friday, April 18, 2014

Python nose testing

Nose provide additional testing functionalities that are not provided in unittests framework. Unlike unittests which is a standard python module you will have to easy_install nose as described here.


use test_mytest.py to name a test module and test_method to name any test methods. If you do so the nosetests will be able to discover and execute those tests automatically.

You will have to use assert_equal(A, B) to test whether A is equal to B.
For numpy arrays use numpy.testing.assert_array_equal instead to compare numpy arrays.

To test the unit tests you must call nosetests as follows.

To display the name of the test functions
nosetests -v test.py

To display the print statements as well
nosetests -v -s test.py

Execute a specific test function within a class in a test module
nosetests -v -s test.py:className:test_method 
 

Friday, February 21, 2014

Synchronized decorator for Python

The following decorator provides synchronization for Python codes.
Reference


def synchronized(lock):
    """ Synchronization decorator. """

    def wrap(f):
        def newFunction(*args, **kw):
            lock.acquire()
            try:
                return f(*args, **kw)
            finally:
                lock.release()
        return newFunction
    return wrap

if __name__ == '__main__':
    from threading import Thread, Lock
    import time

    myLock = Lock()

    class MyThread(Thread):
        def __init__(self, n):
            Thread.__init__(self)
            self.n = n

        @synchronized(myLock)
        def run(self):
            """ Print out some stuff.

            The method sleeps for a second each iteration.  If another thread
            were running, it would execute then.
            But since the only active threads are all synchronized on the same
            lock, no other thread will run.
            """

            for i in range(5):
                print 'Thread %d: Start %d...' % (self.n, i),
                time.sleep(1)
                print '...stop [%d].' % self.n

    threads = [MyThread(i) for i in range(10)]
    for t in threads:
        t.start()

    for t in threads:
        t.join()

Filter forwarded messages in GMail

If you want to filter a message that has been forwarded to your GMail account by one of your other email services, then you can create a filter with Has Words field set to

deliveredto:your_email_service_mail_accountname_that_forwarded_message. 

You can set filters for example to archive such forwarded messages without showing them in your Inbox. You might want to do this for example, if you want to keep a record of all your emails in your GMail account but do not want to display duplicates when you are using an email client that display emails received in multiple accounts.

Thursday, December 19, 2013

numpy dot vs inner

dot and inner returns the same result for 1D arrays.
However, for 2D arrays (matrices), dot gives the matrix product, whereas inner gives the sum-product over the last axis.

References
inner vs. dot
inner document
dot document

what is sum product?
A = [[x1, x2], [x3, x4]]
B = [[y1, y2], [y3, y4]]

sum product of A and B is given by numpy.inner(A,B) as follows
A's row i is element-wise multiplied by B's row j and the values are added to get the (i,j) element

[[x1*y1+x2*y2, x1*y3+x2*y4], [x3*y1+x4*y2, x3*y3+x4*y4]]

Continuously monitor GPU usage

 For nvidia GPUs do the follwing: nvidia-smi -l 1