About Gaurav Vichare

Hi , I am Gaurav : author of this blog , Software engineer and a Photoshop lover . You can connect me on LinkedIn, cirlce me on Google or follow on Twitter

How to assign python variable to Javascript variable in web2py

You might be using following statement to assign python variable to javascript variable on older versions of web2py (<2.9.11), may be because of this question on stackoverflow: http://stackoverflow.com/questions/31216574/how-to-pass-data-from-python-to-javascript-in-web2py

Above statement was working on web2py 2.9.11 and before because of behavioral bug in web2py, which got fixed in web2py 2.9.11. Now response.json()  sets content-type = application/json . So because of above line, on web2py >2.9.11 html code is displayed in browser instead of rendering it as html.

Correct way:

First convert python variable to json in controller and then pass it to view. And in view, use XML() to assign it to javascript variable so that it is not escaped

Controller

View

Other solution is use ASSIGNJS html helper

Controller

View

 

How to change admin app url in web2py?

This question was asked here How to rename “/admin” to other URL in web2py?

For all the web2py applications, web2py admin url is same, i. e. www.example.com/admin or 127.0.0.1:8000/admin. One should prevent access to admin interface  to public or external users. Admin url should be changed from /admin to /some-other-url which is difficult to guess. admin is just another app like welcome or example app, so changing appliacation name will change url to it.

For example to change admin  to w2p-adm2in  do following steps

1. Rename admin folder with w2p-adm2in
On linux

On Windows

If you are in local environment, you can also do it using your file explorer

2. Fix broken links
Due to changes done in Step 1, links to admin app are broken. Links to admin app needs to be replaced with links to w2p-adm2in.

In app_name/controllers/appadmin.py
Update response.menu, change it to following

3. Instead of admin app, check user is logged in to w2p-adm2in app
while accessing appadmin, by default controller app_name/controllers/appadmin.py checks whether user is logged in to admin app or not, if not then it redirects to admin app(admin app login page), now it should check whether user is logged in to w2p-adm2in app and if not it should redirect to w2p-adm2in. To acheive this, pass other_application='w2p-adm2in'  as parameter to method gluon.fileutils.check_credentials()

After step 2 and step 3, diff file of app_name/controllers/appadmin.py will look like following

4. Change links to error page (tickets)
By default links to error is /admin/default/ticket/[ticket_id], Now it should be /w2p-adm2in/default/ticket/[ticket_id]
So add/replace/append following lines to web2py/routes.py

If routes.py file does not exist in web2py folder then create it containing above lines.

 

If you more changes that needs to be done, please comment below.

How to do file validation in web2py and write custom validator to validate files?

In this article, I will explain built in validator IS_IMAGE  and  IS_UPLOAD_FILENAME, also a custom validator.
It is important to validate file uploaded on server to restrict malicious files like .py .php, .exe bat etc.
Web2py provides 2 validators to validate files, IS_IMAGE  and IS_UPLOAD_FILENAME .

  1. IS_IMAGE

It checks whether uploaded file is of type image and file is in any of supported image formats

Following statement Checks if uploaded file is either png or bmp

2. IS_UPLOAD_FILENAME

It checks whether uploaded file meets the given rules for filename and extension.

You can provide regular expressions in filename and extension parameters.
If filename or extension doesn’t match the expression/criteria then it shows error in form.
You can provide your own error message using argument error_message

Example:

If you want to allow all files starting with word ‘log’ and having extension “txt”:

Like IS_IMAGE validator we can not pass list of valid extensions to IS_UPLOAD_FILENAME validator.
But we can achieve this using regular expression or by writing our own validator

  1. Using regular expression:

The above expression will allow all the files having extension pdf, docx, doc or xls.

^ Matches the start of string

$ Matches end of the string

(pdf|docx|doc|xls) match any value from group pdf, docx, doc and xls

Read more about Regular Expression

2. Using custom validator:

Following validator checks whether uploaded file has extension from the valid extension list provided through argument extensions. It will also show error for empty file.

Usage:

Advantage of custom validator is you can add and modify restriction as per your need

If you are facing any issues regarding above post please comment below

How to write custom download controller in web2py?

Suppose we have employee table with resume as uplaod field. And we want to download file on the basis of employee id.
Download  url is like app_name/custom_download/employee_id
or <a href="{{=URL('default', 'custom_download', args = employee_id)}}" download>

example: www.example.com/default/custom_download/1

Consider following employee schema:

Then custom download controller will be:

If you have any queries related to this code, please comment below.

How to password protect PDF file using Python?

We will use python library PyPDF2 to set password to pdf file.

To install PyPDF2:

We are using encrypt function of PyPDF2.

encrypt(user_password, owner_password=None, use_128bit=True)

  • user_password  – The “user password” allows opening and reading the PDF file with the restrictions .
  • owner_password – The “owner password”  have no restrictions. By default, the owner password is the same as the user password.
  • use_128bit  – Decides which encryption to use128bit or 40bit.

If you have any queries, please comment!

Memory profiling in Python using memory_profiler

To install memory_profiler:

    1. Profile function/script:
      Add following line in script to import memory profiler:

      Decorate the function you would like to profile with @profile
      Example:

      Run python script to get memory usage line by line.

      Another method to profile function is , decorate function with @profile and then run script using following command:

      2. Profile web2py application/(External scripts):
      Start webpy server using following command :

      Now open application from browser and open page you want to profile. Then stop web2py server. To view result run command ‘mprof plot’. But to plot result you need package ‘matplotlib’.
      To install matplotlib, first install required packages for matplotlib:

      Now install matplotlib:

      Now run following command to get memory profile graph (Memory Usage vs Time):

      The available commands for mprof :

      • mprof run: running an executable, recording memory usage
      • mprof plot: plotting one the recorded memory usage (by default, the last one)
      • mprof list: listing all recorded memory usage files.
      • mprof clean: delete recorded memory usage files.
      • mprof rm : delete particular  memory usage file.

Time profiling in python using cProfile

In this article we will cover following points:

  1.  Profile single function
  2. Profile complete python script
  3. How to read .prof file
  4. Profile web2py application

To install cprofile on ubuntu use following command:

1) Profile single function

To profile function, import cProfile

Then call function using cProfile.run()

This will write profiling data in test.profile .

Example:

Later in this post we will see how to read .profile file

2) Profile python script

Run python script using following command :

This will profile complete script and write profiling data in test.profile.

3) How to read .profile file?

Using pstats module:

.profile file is in binary format. So to examine data from .profile file use the interactive mode of the pstats module by running it as a script with the profile data file as argument.

Now you will go in interactive profile statistics browser.

Now use stats command to display stats

You can sort stats using following sort keys:
cumulative — cumulative time
module — file name
ncalls — call count
pcalls — primitive call count
file — file name
line — line number
name — function name
calls — call count
stdname — standard name
nfl — name/file/line
filename — file name
cumtime — cumulative time
time — internal time
tottime — internal time

For example:

Above command will sort records using tottime ( internal time)

To show particular number of records use command “stats count”

For example:

This will show only 10 records.

Other method to read .profile file is using command “cprofilev -f test_func.profile”

Run above command in terminal and view stats in browser on http://127.0.0.1:4000

Advantage of this method is you can easily sort records ,just by one click on column name.

4. Profile web2py application

Create a empty text file in a directory and then pass this text file as argument to web2py. All profiling data will be stored in this file.
For example :
Create lcm.txt in ‘/home/gaurav/temp/lcm/” directory. Now run web2py server using following command with command line argument -F (profiler filename)

Note: For some web2py version , we have to provide directory using -f (not -F) option and data will be written in that directory in the form of .profile files. We can read these files using above mentioned methods.

Now open application from browser and open pages you want to profile.Then stop web2py server. This will write profiling data to lcm.txt. Data in text file will be in readable format , no need of pstats module to read data.

How to measure performance of code ? – Basic methods

In this article, we will cover some very basic methods to calculate performance of script. In next article we will cover advance tool like cProfile and memory_profiler.

What is profiling?

  • Profiling is program analysis that measures, for example, the space (memory) or time complexity of a program, the usage of particular instructions, or the frequency and duration of function calls.
  • Profiling or performance measurement is main tool for code optimization.
  • There are two types of profiling : Time profiling and Memory profiling
  • We should also consider memory leaks to optimize code.

1) Date

Run script by running date command before and after script to find time when script started execution and when it completed.

This prints time before running script and again print time when execution is complete. So we can find execution time by subtracting end time from start time.

2) time

Unix time command is helpful to figure out how much time command, shell script or python program takes for execution.
Usage:

output:

  • User : CPU time spent in user-mode (outside the kernel)
  • System : CPU time spent in the kernel
  • CPU : Percent of CPU this job got
  • Total : elapsed time, start to end

Why user + system ≠ total ?
total represents actual elapsed time, while user and sys values represent CPU execution time.

3) /usr/bin/time

/usr/bin/time gives more detailed output than ‘time‘.
Usage:

Output:

What is difference between usr/bin/time and time ?
time is built in unix command and usr/bin/time is GNU time

Size Contest (SPOJ Challenge problem 378)

Given the set of integers, find the sum of all positive integers in it. Solutions can be sent in any language supported by SPOJ except Whitespace.

Input

t – number of test cases [t < 1000]
On each of next t lines given a integer N [-1000 <= N <= 1000]

Output

One integer equals to sum of all positive integers.

Score

Score equals to size of source code of your program except symbols with ASCII code <= 32.

Example

C Program :

 

Factorial (SPOJ problem 11)

View problem on SPOJ

The most important part of a GSM network is so called Base Transceiver Station (BTS). These transceivers form the areas called cells (this term gave the name to the cellular phone) and every phone connects to the BTS with the strongest signal (in a little simplified view). Of course, BTSes need some attention and technicians need to check their function periodically.

ACM technicians faced a very interesting problem recently. Given a set of BTSes to visit, they needed to find the shortest path to visit all of the given points and return back to the central company building. Programmers have spent several months studying this problem but with no results. They were unable to find the solution fast enough. After a long time, one of the programmers found this problem in a conference article. Unfortunately, he found that the problem is so called “Travelling Salesman Problem” and it is very hard to solve. If we have N BTSes to be visited, we can visit them in any order, giving us N! possibilities to examine. The function expressing that number is called factorial and can be computed as a product 1.2.3.4….N. The number is very high even for a relatively small N.

The programmers understood they had no chance to solve the problem. But because they have already received the research grant from the government, they needed to continue with their studies and produce at least some results. So they started to study behaviour of the factorial function.

For example, they defined the function Z. For any positive integer NZ(N) is the number of zeros at the end of the decimal form of number N!. They noticed that this function never decreases. If we have two numbersN1<N2, then Z(N1) <= Z(N2). It is because we can never “lose” any trailing zero by multiplying by any positive number. We can only get new and new zeros. The function Z is very interesting, so we need a computer program that can determine its value efficiently.

Input

There is a single positive integer T on the first line of input (equal to about 100000). It stands for the number of numbers to follow. Then there are T lines, each containing exactly one positive integer number N, 1 <= N<= 1000000000.

Output

For every number N, output a single line containing the single non-negative integer Z(N).

Example

Sample Input:

6
3
60
100
1024
23456
8735373
Sample Output:

0
14
24
253
5861
2183837

 Program: