Thursday 27 November 2014

How to test file upload action in Play Framework 2 in Java?

In this post, we will see how we can test an action that expects a file upload (e.g. a profile picture or an image) in Play Framework 2 for Java. First, lets create an action that accepts a file upload as below:

 public static F.Promise uploadProfileImage() {
    final Http.MultipartFormData body = request().body().asMultipartFormData();
    final Http.MultipartFormData.FilePart picture = body.getFile("picture");
    User user = // get the user from db
    
    final File imgFile = picture.getFile();
    final String imgPathToSave = "images/" + "some unique name";
    //save on disk
    final boolean success = new File("images").mkdirs();
    final byte[] bytes = IOUtils.toByteArray(new FileInputStream(imgFile));
    FileUtils.writeByteArrayToFile(new File(imgPathToSave),bytes);
    user.profileImageUrl = imgPathToSave;
    user.save();
       
}


AS you can see, the body of the request is a MultipartFormData. This makes a bit tricky to write functional tests in Play Framework 2 for Java actually. But, let's see the solution.

@Test
public void uploadProfileImageOK() throws Exception { 
   MultipartFormData.FilePart part = new MultipartFormData.FilePart<>("picture","test-image.jpg",Scala.Option("image/jpeg"),new File("test-image.jpg"));

   MultipartFormData formData = new MultipartFormData(Scala.>emptyMap(),Scala.toSeq(Arrays.asList(part)),Scala.emptySeq(),Scala. emptySeq());

   AnyContent anyContent = new AnyContentAsMultipartFormData(formData);

   Result uploadImageResult = callAction(controllers.routes.ref.UserController.uploadProfileImage(), fakeRequest().withHeader("Csrf-Token", "nocheck").withCookies(fakeCookie).withAnyContent(anyContent,"multipart/form-data", "POST"));

   assertEquals(OK, status(uploadImageResult));

   
}

To see more about functional testing in Play Framework 2 using Java see: How to test an action secure by SecureSocial in Play Framework 2 in Java

How to test an action secure by SecureSocial in Play Framework 2 in Java?

SecureSocial is an authentication module for Play Framework 2 applications supporting OAuth, OAuth2, OpenID, Username/Password and custom authentication schemes. Suppose you have an action and you have secured it with @SecureSocial.SecuredAction annotation:

public class Application extends Controller {
    @SecureSocial.SecuredAction
    public static Result index() {
        Identity user = (Identity) ctx().args.get(SecureSocial.USER_KEY);
        return ok(index.render(user));
    }
}

At the time of writing this blog post, SecureSocial only uses cookies for authentication. If you want to write a functional test for this action without taking care of a fake cookie you will get an unauthorized message. To do this we need to some provisioning before our test. Best is to use the @Befor. Below is the steps:
  1. Lets assume that you save your user authentication data in the User model. Here we create a fake user before
  2. 
    @Before
    public void setUp() throws Exception {
       User user = new User();
       User user = new User();
       user.firstName = "jack";
       user.lastName = "sparrow";
       user.email = "jack.sparrow@caribbean.com";
       user.providerId = "userpasswordid";
       user.password = "$2a$10$ywqls6dRsN4wLr.xNydi2uDVFNkOlmi9WSAfRy.RXdN5sgKKnKhau";
       user.authMethod = "userPassword";
       user.save();
    }
    
    
    Please note that the password that you see above is a hash produced with "Bcrypt" encryption and the value depends on how you have implemented this. See Secure Social Password Plugin for more info.
  3. Now create a class named FunctionalTestHelpers and add the following method:
  4. 
    public static Http.Cookie getFakeCookie(String email){
            User user = User.findByEmail(email);
    
            SocialUser socialUser = new SocialUser(new IdentityId(user.email,user.providerId),
                    user.firstName,
                    user.lastName,
                    String.format("%s %s", user.firstName, user.lastName),
                    Option.apply(user.email),
                    null,
                    new AuthenticationMethod("userPassword"),
                    null,
                    null,
                    Some.apply(new PasswordInfo("bcrypt", user.password, null))
            );
    
            Either either = Authenticator.create(socialUser);
            Authenticator auth = (Authenticator) either.right().get();
            Cookie scalaCookie = auth.toCookie();
    
            return new Http.Cookie(scalaCookie.name(),
                                   scalaCookie.value(),
                                   null,
                                   scalaCookie.path(),
                                   null,
                                   scalaCookie.secure(),
                                   scalaCookie.httpOnly());
    
        }
    
    
  5. Now you can use the code below in your tests to get a fake cookie. The code above basically mimics what SecureSocial does for creating a cookie.
  6.  
    Http.Cookie fakeCookie = FunctionalTestHelpers.getFakeCookie("jack.sparrow@caribbean.com"); 
    
  7. Now lets see it in action:
  8. 
    final Result deleteResult = callAction(controllers.routes.ref.TaskController.removeTask(taskId),fakeRequest().withCookies(fakeCookie).withHeader("Csrf-Token", "nocheck"));
    assertThat(status(deleteResult)).isEqualTo(OK);
    
    

Wednesday 26 November 2014

How to test actions annotated with @RequireCSRFCheck in Play Framework 2?

Let say you have an action as below that you want to protect against CSRF attacks, so you add the @RequireCSRFCheck annotation:

@RequireCSRFCheck
public Result saveUser() {
    // Handle body (process a form)
    return ok();
}

Now suppose you want to write some functional tests for this action. All you need to do is to add fake "nocheck" to your header in "callAction" as below:

final Result result = callAction(controllers.routes.ref.UserController.saveUser(),fakeRequest().withHeader("Csrf-Token", "nocheck"));
assertThat(status(result)).isEqualTo(OK);

For more information about CSRF checks in Play Framework 2 see: JavaCsrf

Tuesday 25 November 2014

Importing a CSV file into a SQLite3 database table in Python

For this you need to import csv, sqlite3 libraries:

import csv, sqlite3

Lets use an in-memory database and create a simple table with two columns:

con = sqlite3.connect(":memory:")
cur = con.cursor()
cur.execute("CREATE TABLE t (col1, col2);")

Now, lets load our .csv file into a dictionary:

with open('data.csv','rb') as fin:
    dr = csv.DictReader(fin) # comma is default delimiter
    to_db = [(i['col1'], i['col2']) for i in dr]

csv.DictReader uses first line in the .csv file for column headings by default. Comma is also the default delimiter. The actual import:

cur.executemany("INSERT INTO t (col1, col2) VALUES (?, ?);", to_db)
con.commit()

If you want to play around with an example, you can try this:

#!/usr/bin/python
# -*- coding: utf-8 -*-

import sys, csv, sqlite3

def main():
    con = sqlite3.connect(sys.argv[1]) # database file input
    cur = con.cursor()
    cur.executescript("""
        DROP TABLE IF EXISTS t;
        CREATE TABLE t (COL1 TEXT, COL2 TEXT);
        """) # checks to see if table exists and makes a fresh table.

    with open(sys.argv[2], "rb") as f: # CSV file input
        reader = csv.reader(f, delimiter=',') # no header information with delimiter
        for row in reader:
            to_db = [unicode(row[0], "utf8"), unicode(row[1], "utf8")] # Appends data from CSV file representing and handling of text
            cur.execute("INSERT INTO neto (COL1, COL2) VALUES(?, ?);", to_db)
            con.commit()
    con.close() # closes connection to database

if __name__=='__main__':
    main()

Please not that the code above, we encode the input.

Preventing the modification of a private field in Java

Let's say you have a class like:

public class Test
{
  private String[] arr = new String[]{"1","2"};    

  public String[] getArr() 
  {
    return arr;
  }
}

Now, I have another class that uses the above class:

Test test = new Test();
test.getArr()[0] ="some value!"; //!!!

Bam! We were able to modify a private field outside of the class. This not what you want! To avoid this situation, you can use of the following solutions:
  1. Either create a defensive copy:
  2. 
    public String[] getArr() {
      return arr == null ? null : Arrays.copyOf(arr, arr.length);
    }
    
    
  3. If you can use a List instead of an array, Collections provides an unmodifiable list:
  4. 
    public List getList() {
        return Collections.unmodifiableList(list);
    }
    
    

What is the use of “assert” in Python?

The
assert
statement/function exists in mostly every other programming language out there. When you do...

assert a_condition

... you're telling the program to test that condition, and trigger an error if the condition is false. In Python, it's roughly equivalent to this:

if not condition:
    raise AssertionError()

Try it in the Python shell:

>>> assert True
>>> assert False
Traceback (most recent call last):
  File "", line 1, in 
AssertionError

Assertions can include an optional message, and you can disable them when you're done debugging. See here for the relevant documentation.

What is a percolator in Elasticsearch?

What you usually do is index documents and get them back by querying. What the percolator allows you to do in a nutshell is index your queries and percolate documents against the indexed queries to know which queries they match. 
It's also called reversed search, as what you do is the opposite to what you are used to. 
 There are different use-cases for the percolator, the first one being any platform that stores users interests in order to send the right content to the right users as soon as it comes in. For instance a user subscribes to a specific topic, and as soon as a new article for that topic comes in, a notification will be sent to the interested users. 
You can express the users interests as an elasticsearch query, using the query DSL, and you can register it in elasticsearch as it was a document. Every time a new article is issued, without needing to index it, you can percolate it to know which users are interested in it. At this point in time you know who needs to receive a notification containing the article link (sending the notification is not done by elasticsearch though). An additional step would also be to index the content itself but that is not required.

How to set specific logging for tests in Play Framework 2 ?

Specifying a different logging configuration is very simple.

Play Framework 2's default logging is logback. To set specific logging configurations for tests follow the steps below:
  1. Just create a logback configuration file (eg test-logger.xml):
  2. 
    <configuration>
    
     <conversionRule conversionWord="coloredLevel" converterClass="play.api.Logger$ColoredLevel" />
    
     <appender name="FILE" class="ch.qos.logback.core.FileAppender">
     <file>${application.home}/logs/application.log</file>
     <encoder>
     <pattern>%date - [%level] - from %logger in %thread %n%message%n%xException%n</pattern>
     </encoder>
     </appender>
    
     <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
     <encoder>
     <pattern>%coloredLevel %logger{15} - %message%n%xException{5}</pattern>
     </encoder>
     </appender>
    
     <logger name="play" level="INFO" />
     <logger name="application" level="DEBUG" />
    
     <!-- Off these ones as they are annoying, and anyway we manage configuration ourself -->
     <logger name="com.avaje.ebean.config.PropertyMapLoader" level="OFF" />
     <logger name="com.avaje.ebeaninternal.server.core.XmlConfigLoader" level="OFF" />
     <logger name="com.avaje.ebeaninternal.server.lib.BackgroundThread" level="OFF" />
     <logger name="com.gargoylesoftware.htmlunit.javascript" level="OFF" />
    
     <root level="ERROR">
     <appender-ref ref="STDOUT" />
     <appender-ref ref="FILE" />
     </root>
    
    </configuration>
    
    
  3. Go to root project folder, open
    build.sbt
    file and add:
    
    javaOptions in Test +="-Dlogger.resource=test-logger.xml"
    
    
    Now whenever you run your tests, the logging configuration written in
    test-logger.xml
    will be read.