Friday, December 27, 2013

Java pattern matching using GLOB patterns (* and ?)

Java has partial glob support since java 7 filePath, but truly glob support I could only find in the good, old, retired, jakarta-oro lib.

Here goes a simple example of how to use it

for(String regex:regexes){
  
    //it's the unix GLOB regex. Java itself does not support it
    //but the good old jakarta oro does
    GlobCompiler compiler=new GlobCompiler();
    Perl5Matcher matcher=new Perl5Matcher();
  
    //there are some interesting options you may want to try
    //org.apache.oro.text.regex.Pattern pat = compiler.compile(regex,GlobCompiler.QUESTION_MATCHES_ZERO_OR_ONE_MASK );
    org.apache.oro.text.regex.Pattern pat = compiler.compile(regex.trim());
    if (site != null && matcher.matches(site, pat)){
        passwordPattern = pattern;
        break;
    }
}

Tuesday, December 17, 2013

Primefaces: capturing the target list item clicked

First of all, kudos to Hatem Alimam for his answer, who solved this problem.

Here's the problem.

I have a @ViewScoped managed bean with a p:pickList filtered by a p:selectMany component using a p:ajax update event.

Like this

<p:outputLabel for="siteFilter" value="Site Filter:" style="width:100px;"/>
<p:selectOneMenu converter="#{siteTypeConverter}" id="siteFilter" value="#{myMB.selectedSiteType}" style="width:200px;">              
   <f:selectItem itemLabel="Select Site Type" itemValue="#{null}" />
   <f:selectItems value="#{myMB.siteTypeFilterList}" var="st" itemLabel="#{st.name}" itemValue="#{st}" />
   <p:ajax update="sites" listener="#{myMB.handleSiteTypeFilterChange}" oncomplete="rebindClicks()"/>
</p:selectOneMenu>

<p:outputLabel for="sites" value="Sites:" style="width:100px;"/>
<p:pickList
   widgetVar="xyzabc"
   id="sites"
   value="#{myMB.sites}"
   var="site"                 
   converter="#{siteConverter}"
   itemLabel="#{site.siteType.name}.#{site.name}"
   itemValue="#{site}" /> 

The idea here is to trigger a new event when the user clicks on some pickList target list item.

So, as suggested by @Hatem, we have to bind the click event to each one of the target elements. Like this

(...)
   <p:remoteCommand name="updateCommand" action="#{myMB.updateAccounts}" update="accounts"/>
</h:form>
<h:outputScript library="js" name="pickListHack.js" />   
(...)

Notice that I had to add it AFTER the form.

and pickListHack.js is located at WebContent/resources/js/pickListHack.js

function rebindClicks() {
    xyzabc.jq.on('click','.ui-picklist-target li',function(){
        var item = $(this).attr('data-item-value');
        alert(item);
        updateCommand([{name:'param', value:item}]);
   });
};

$(document).ready(function() {
    rebindClicks();
});

The trick here seems to be that, after the update event, the binds are lost, so we have to rebind them after the update. That's  why p:ajax has that oncomplete event.

Finally, when the user clicks on the element from the target list, we call the updateCommand, declared via p:remoteCommand.

Notice that the selected item is passed as a parameter called "param". We get this parameter back in the managed bean like this

public void updateAccounts(){
   String value = FacesContext.getCurrentInstance().getExternalContext().getRequestParameterMap().get("param");
   System.out.println(">"+value);
   this.accounts = value;
}

In this case, the value is just the ID, because that's what my siteConverter does to the object.

@ManagedBean
@RequestScoped
public class SiteConverter implements Converter,Serializable{

    private static final long serialVersionUID = -194442504109002565L;

    @EJB
    private MyEJB myEJB;

    @Override
    public Object getAsObject(FacesContext arg0, UIComponent arg1, String arg2)
            throws ConverterException {
        if (arg2 != null){
            Site s = myEJB.getSiteById(Long.parseLong(arg2));
            return s;
        }else{
            return null;
        }
    }

    @Override
    public String getAsString(FacesContext arg0, UIComponent arg1, Object arg2)
            throws ConverterException {
        if (arg2 != null){
            Site s = (Site)arg2;
            return String.valueOf(s.getId());
        }else{
            return null;
        }
    }
}



Monday, December 16, 2013

Java, expect and groovy

If you need the TCL expect functionality in your java software, you basically have 4 options


TCL-Java bridges such as Jacl/TclBlend (http://tcljava.sourceforge.net) are useful to run small pieces of TCL code, but since expect contains some native C libraries, jacl is not an option. In the best case, you can write a TCL wrapper with TclBlend that calls your java code instead of the opposite.

So, from these 4 options, only ExpectJ supports expect "interact" mode. In expect, the "interact" command gives back the session control to the user. So, if the idea is to connect to a remote site after multiple hops, and get the session back (instead of running fire-and-forget scripts), ExpectJ is the way to go.

Now, you can also expose ExpectJ directives in a nice way (DSL) using groove. This is specially useful if you want to let the user write his/her own "expect" scripts.

Allowing the user to run his/her own scripts from inside an application also has security issues. For example, would you allow the user to add a "system.exit()" command? Not a good idea. We have to limit this too.

Groovy has some syntactic sugar features that lets the developer to write DSLs in a much faster and simpler way that it would be if you decide to write a full language from the scratch, with tools like ANTLR and JavaCC. You can also use eclipse to edit and run your Groovy code in a nice IDE and the learning curve, for Java developers, is nice.

To demonstrate a simple proof-of-concept, let's assume these 4 steps

  • Wrap ExpectJ engine in a java class
  • Wrap java class into a Groovy class
  • Add Groovy syntatic sugar
  • Sandbox Groovy script interpreter

We're also using Jsch (http://www.jcraft.com/jsch/), a pure java-based SSH client.

Step #1 - Wrap ExpectJ engine in a java class

    public class Expect {
       
        private ExpectJ expectinator;
        private Spawn spawn;
        private int lastPos;

        public Expect() throws IOException{
            expectinator = new ExpectJ(5);
            spawn = expectinator.spawn("localhost", 22, "leoks", "xyz");
        }
       
        public void send(String s) throws IOException{
            spawn.send(s+"\n");
        }
       
        public String expect(String s) throws IOException, TimeoutException{
            spawn.expect(s);
            return getReply();
        }
       
        private String getReply(){
            String all = spawn.getCurrentStandardOutContents();
            int newLastPos = all.length();
            String reply = all.substring(lastPos);
            lastPos = newLastPos;
            return reply;
        }
       
    }

Step #2 - Wrap java class into a Groovy class

    class ExpectGroovy {
       
        def Expect expectJava = new Expect()
       
        def expect(String expression){
            expectJava.expect(expression)
        }
       
        def send(String command){
            return expectJava.send(command)
        }
    }

Step #3 - Add Groovy syntatic sugar

    class Example {
        static main(args) {
            def binding = new Binding(exp: new ExpectGroovy())
           
            ExpectSandbox sandbox = new ExpectSandbox();
           
            def config = new CompilerConfiguration()
            config.scriptBaseClass = ExpectBaseScriptClass.class.name
            config.addCompilationCustomizers(new SandboxTransformer())
            sandbox.register()
           
            def shell = new GroovyShell(this.class.classLoader, binding, config)
            String script = ...
            try{
                shell.evaluate(script)
            }catch(e){e.printStackTrace()}
           
            sandbox.unregister()
            print "OK"
        }
    }
    abstract class ExpectBaseScriptClass extends Script{
        void expect(String expression){
            this.binding.exp.expect expression
        }
       
        void send(String command){
            this.binding.exp.send command
        }
    }

Step #4 - Sandbox Groovy script interpreter

Now, the groovy script may look like this

    expect "\\$"

    //try to uncomment - the script does not kill the process, instead this instruction
    //is blocked by groovy security restrictions
    //see http://groovy.codehaus.org/Advanced+compiler+configuration
    //System.exit(0)

    x = 0
    while (x < 5){
        result = send "hostname"
        expect "\\$"
        x++
    }

Please notice that

  •     The “expect” object is implicit (binding)
  •     Groovy loops and variable assignment for free (mallet for example needed a special code just for loop evaluation)
  •     Groovy allows to restrict certain undesired commands, as well as java packages restrictions
  •     No explicit grammar defined
  •     Groovy can be coded with eclipse, with code completion, etc

References


Thursday, November 21, 2013

How to monitor your tomee+ app running at eclipse indigo

It's so sad that Eclipse dropped its support to TPTP project, so it's not available for Indigo or June. MAT is not exactly something friendly. So, the easiest way to monitor the heap I've found was to use GCViewer. It's simple, it has a button to refresh the graph on every file change (it parses the garbage collector file and presents in a nice graph) and it really helps.

So, download the GCViewer fork (that will allow you to parse the most recent JVM gc files) and add these to your tomee server launch configuration parameters

-Xloggc:gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+HeapDumpOnOutOfMemoryError

Just notice that when the server shuts down, the gc file vanishes too.

I wonder why Eclipse can't integrate such a simple tool.

Upgrading from TomEE+ 1.5.2 to TomEE+ 1.6.0

It's getting to an end. My impressions

  • I had to recreate JMS tables at Oracle, or you'll get a mysterious ORA-01465: invalid hex number.
  • I had to update Quartz DB Schema adding a new column as described at http://quartz-scheduler.org/documentation/quartz-2.2.x/new-in-quartz-2_2  . They say "ALTER TABLE QRTZ_FIRED_TRIGGERS ADD COLUMN SCHED_TIME NUMBER(13) NOT NULL;" but this query seems not to work, so just open sqldeveloper and create the column by hand.
  • OpenJPA seems to have a bug related to embedded composite keys, see http://stackoverflow.com/questions/20123910/openjpa-persistenceexception-while-trying-to-save-class-with-composite-primary , so I'll have to write more code to use a surrogate key + unique clause for the original composite key. Lucky me I only use a few (it seems that composite primary keys in JPA are some sort of old-fashioned anyway) otherwise this would be a showstopper.
  • I had to change my applications.properties file to replace the driver to driverDelegateClass = org.quartz.impl.jdbcjobstore.oracle.OracleDelegate instead of the standard one.
  • Performance was significantly improved from 1.5.2 to 1.6.0 but it's early to say it's more stable. At least, seems to be better and apache developers seem to be focusing on 1.6.0, so several fixes are being addressed in this version. 1.5.2 seems to be receiving less attention, even for 1.5.3, I hear almost nothing about it. So it seems 1.6.0 is "de facto" version. Milestones would be nice. Working only with snapshots gives me the sensation that I am developing in a slack line.
  • When people say TomEE community is active, they're probably meaning that Romain is there. No one else answers the questions as fast as him. The same about OpenJPA and Rick.


Thursday, October 24, 2013

How to get TCL "huddle" from inside JACL

JACL can parse the "old TCL" dict format.

TCL has a very nice library to deal with this kind of format, called Huddle. Huddle is part of the tcllib package and, specifically, a very useful command called "compile" was added after version 1.11 (I don't know exactly which version, but it's before 1.15).

Huddle relies on another TCL library called "dict". This library was incorporated into the language since TCL 8.5.

The problem here now is Jacl, the java TCL interpreter. It was not build with "dict" support, so you need a patch to include the "dict" command (http://sourceforge.net/p/tcljava/patches/15).

Finally, jacl identifies itself as "less than TCL 8.5", and the first thing that huddle checks is if it needs "dict". See $PATHTO/tcllib1.15/yaml/huddle.tcl

right in the beginning, you'll see
if { [package vcompare [package provide Tcl] 8.5] < 0 } {
    package require dict
}

to avoid the annoying message of "Exception in thread "main" tcl.lang.TclException:
can't find package dict", you MUST comment this part of the huddle code, so it will be just like this

if { [package vcompare [package provide Tcl] 8.5] < 0 } {
#    package require dict
}

Then you'll be able to write java code just like this

        String x = "{username {root}} {shadow {x}}";
        interp.eval("lappend auto_path \"LIB\"".replaceAll("LIB", "/PATH/TO/tcllib1.15/"));     
        interp.eval("package require huddle");
        interp.eval("set x [ huddle compile { list { dict } }  {"+x+"} ]");
        interp.eval("set i [ huddle jsondump $x ]");
        String json = interp.getVar("i", 0).toString();
        System.out.println(json);

to get this kind of result

[
  {"username": "root"},
  {"shadow": "x"}
]

ps. Tom kindly answered my email at the tcl/java users mailing list, suggesting a cleaner and more elegant solution for this, that I'll certainly evaluate. Thanks Tom!

Friday, October 11, 2013

I cannot audit with TomEE, so what should I do?

Basic CRUD operations audit, even being an old problem of many apps, is still unresolved by most java frameworks (just as reloadable config files), which gives me the sensation that we, programmers, are sometimes running in circles.

So I've filled a bug for TomEE (probably related to eclipse and OSGI?) because openJPA @Auditable seems not to work on any tomee distribution I can test (1.5.2, 1.5.3, 1.6.0). The sample code you can get at the bug report.

What is the workaround?

Interceptors to the rescue.

First, add this to your src/META-INF/beans.xml

<beans>
  <interceptors>
    <class>AuditInterceptor</class>
  </interceptors>
</beans>


To implement the interceptor, you need an annotation and the interceptor itself

@InterceptorBinding
@Target({TYPE, METHOD})
@Retention(RUNTIME)
public @interface Audit {
}


and


@Interceptor
@Audit
public class AuditInterceptor implements Serializable{

    private static final long serialVersionUID = 6988858067583207506L;
   
    @EJB
    private LogManagerEJB logManagerEJB;

    private JSONSerializer json = new JSONSerializer();

    @AroundInvoke
    public Object logMethodEntry(InvocationContext ctx) throws Exception {
       
        String action = ctx.getMethod().getDeclaringClass()+"."+ctx.getMethod().getName();
        String subject = json.serialize(ctx.getParameters());
        logManagerEJB.saveTelemetryLogs(action, subject);

        return ctx.proceed();
    }
}


and logManagerEJB looks like

    public void saveTelemetryLogs(String action, String subject) {
        TelemetryLog telemetryLog = new TelemetryLog();
        telemetryLog.setAction(action);
        telemetryLog.setActor(JSFUtil.getAuthenticatedUser().getUsername());
        telemetryLog.setSubject(subject);
        telemetryLog.setTs(new Timestamp(System.currentTimeMillis()));
        this.baseService.getTelemetryLogDAO().add(telemetryLog);
    }


DAO is omitted because it's just entity.persist() on the object.

Then you add the @Audit annotation for all methods you want to audit (usually in your business logic layer)

    @Audit
    public void saveUserGroupAccess(UserGroupAccessPK pk) {
        UserGroupAccess uga = new UserGroupAccess();
        uga.setPk(pk);
        this.baseService.getUserGroupAccessDAO().add(uga );       
    }





Wednesday, October 9, 2013

RUP humps are not that accurate

Actually, it was never intended to be, as many can think :-)

http://en.wikipedia.org/wiki/RUP_hump

(...)Over the years, this diagram has become increasingly connected with RUP, so that sometimes it is regarded as a logo for the process. The chart has then since been spread widely over the Internet. A known misconception about the hump chart is, that it is based on empirical assessment of actual projects rather than on Kruchten's educated guess.
...I always insisted that these humps were just illustrative, as well as the number and duration of iterations shown on the horizontal axis, but many people wanted to read much more meaning in that diagram than I intended. For example, a gentleman from Korea once wrote me to ask for a large original diagram to measure the heights, and ‘integrate' the area under the humps, to help him do project estimation...(...)

 We use to call them "the whales" :-)


Thursday, October 3, 2013

JSF Generic Searchable Table with Primefaces

Just because http://www.primefaces.org/showcase/ui/datatableDynamicColumns.jsf is just "a little" over for me.

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"
    xmlns:f="http://java.sun.com/jsf/core"
    xmlns:h="http://java.sun.com/jsf/html"
    xmlns:ui="http://java.sun.com/jsf/facelets"
    xmlns:p="http://primefaces.org/ui">
<h:head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <title>Search</title>
</h:head>
<h:body>
    <h:form id="sm">
        <p:spacer height="10" />
        <p:fieldset legend="Search">
        <p:panel>
            <p:dataTable
                var="dataRow"
                value="#{MB.data}"
                paginator="true" rows="10" 
                paginatorTemplate="{RowsPerPageDropdown} {FirstPageLink} {PreviousPageLink} {CurrentPageReport} {NextPageLink} {LastPageLink}" 
                rowsPerPageTemplate="5,10,15"   
                   emptyMessage="No sites found with given criteria"
                   filteredValue="#{MB.filteredData}"
                   sortBy="#{MB.defaultColumnSort}"
                id="rowsTable">
                
                <p:columns value="#{MB.columns}" var="column" columnIndexVar="colIndex" sortBy="#{column}" filterBy="#{column}" filterMatchMode="contains"> 
                    <f:facet name="header">#{column}</f:facet> 
                    #{dataRow[column]} 
                   </p:columns>
                    
           </p:dataTable>
        </p:panel>
        </p:fieldset>
           
    </h:form>
</h:body>
</html>


and the MB goes like this

import java.io.Serializable;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;

import javax.annotation.PostConstruct;
import javax.ejb.EJB;
import javax.faces.bean.ManagedBean;
import javax.faces.bean.SessionScoped;

@ManagedBean
@SessionScoped
public class MB implements Serializable {
   
    private static final long serialVersionUID = -3964302189254255157L;
   
    private List<String> columns = new ArrayList<String>();
   
    private List<Map<String,String>> data;
   
    private List<Map<String,String>> filteredData;
   
    private String defaultColumnSort;

    public void clear() {
        data = null;
        setFilteredData(null);
    }

    @EJB
    private EJB ejb;
   
    @PostConstruct
    public void init() {
        try {
            open("xyz");
        } catch (Exception e) {
            e.printStackTrace();
            JSFUtil.addErrorMessage("Could not list");
        }
    }
   

    /**
     * entry point
     *
     * @return
     */
    public String open(String table){
        try {
            columns = ejb.getTableColumns(table);
            data = ejb.getTable(table);
            filteredData = data;
            defaultColumnSort = columns.get(0);
            return "thispage";
        } catch (Exception e) {
            e.printStackTrace();
            JSFUtil.addErrorMessage("Could not open");
        }
        return null;
    }
   
    public List<Map<String, String>> getData() {
        return data;
    }

    public void setData(List<Map<String, String>> data) {
        this.data = data;
    }


    public List<Map<String,String>> getFilteredData() {
        return filteredData;
    }


    public void setFilteredData(List<Map<String,String>> filteredData) {
        this.filteredData = filteredData;
    }


    public List<String> getColumns() {
        return columns;
    }


    public void setColumns(List<String> columns) {
        this.columns = columns;
    }


    public String getDefaultColumnSort() {
        return defaultColumnSort;
    }


    public void setDefaultColumnSort(String defaultColumnSort) {
        this.defaultColumnSort = defaultColumnSort;
    }
}

Monday, September 23, 2013

Monday's Useful Link's

  1. http://www.artfulsoftware.com/infotree/queries.php - SQL queries cookbook
  2. http://txt2re.com/ - user-friendly regex generator

Wednesday, September 18, 2013

(yet another) How to enhance OpenJPA classes with TomEE 1.5.2 using Eclipse

import java.io.File;
import java.util.Collection;

import org.apache.commons.io.FileUtils;

/**
 * OpenJPA relies on a class "enhancement" process that is not fully supported in a easy way for eclipse users.
 *
 * <p>Before writing this script, I've considered the following actions
 *
 * <ol>
 * <li>Trying the <a href="http://openjpa.apache.org/openjpaeclipseinstallation.html">eclipse openJPA plugin</a> - the plugin UI hides the last
 * check button and seems not to work properly.
 * <li>Trying the <a href="http://openjpa.apache.org/enhancement-with-eclipse.html">recommended ant builder</a>. I would have to change the
 * project structure
 * <li>Adding <a href="http://webspherepersistence.blogspot.com.br/2009/02/openjpa-enhancement.html">a user-agent parameter to the VM</a>
 * - but at the time of this writing, TomEE 1.5.2 (the "stable" one) had a bug that the javaagent doesn't work (according to Romain), so
 * they fixed in the 1.6.0 trunk, which does not seems acceptable for any project that goes in production for a client.
 * <li>Switching to IBM JDK 7 (J9) virtual machines. Although they default to subclass for enhancement, which is not recommended, the
 * system seems to work with them, for both Windows and Linux, but the gotcha here is that for Windows, IBM distributes the JDK in a
 * "<a href="http://www.ibm.com/developerworks/java/jdk/eclipse/">IBM Development Package for Eclipse</a>" which deals with the problem gracefully
 * . I consider this as an acceptable alternative, but remember that using the IBM JDK is no the same
 * as actually enhancing the classes, as recommended by OpenJPA.
 * </ol>
 *
 * <p>So please forgive me if this procedure is cumbersome.
 *
 * <p>Now, before running standalone tests and migration procedures from main() methods or from JUnit @Test annotated methods, you must
 * run this procedure below to enhance the classes. The same for the web app, but using the destination path used by the server you're
 * running from inside eclipse. And of course, before deploying...
 *
 * <p>September 2013
 *
 *
 */
public class Enhancer {

    public static void main(String[] args) {

        String baseClassDir = "/path/to/eclipse/workspace/.metadata/.plugins/org.eclipse.wst.server.core/tmp0/wtpwebapps/yourapp/WEB-INF/classes/";
//        String baseClassDir = "/path/to/eclipse/workspace/yourapp/build/classes/";
        Collection<File> files = FileUtils.listFiles(new File(baseClassDir), null, true);
        for(File file:files){
            String[] a = new String[3];
            a[0] = "-d";
            a[1] = baseClassDir;
            a[2] = file.getAbsolutePath();
            org.apache.openjpa.enhance.PCEnhancer.main(a);
        }
    }

}

Wednesday, September 4, 2013

How to run a SQL query against a in-memory CSV file

It's a pity that CsvJdbc cannot run against an in-memory CSV string or InputStream or Reader. That's because it follows the JDBC contract and the only way to push data inside the driver is using the JDBC url connection string.

Too bad, because even a JDBC url does not have any built-in method to return a map of parameters (as far as I know) and other URL functions used to do the same thing for http urls don't work because "jdbc" is not a valid protocol ;-).

So, it's time to do nasty things using typecasting and altering the source code. Something like this.

[1] Replace org.relique.jdbc.csv.CsvConnection with another version that includes an extra attribute, a Reader

public class CsvConnection implements Connection{
...
private Reader reader;
...
    public Reader getReader() {
        return reader;
    }

    public void setReader(Reader reader) {
        this.reader = reader;
    }
}


[2] Now, create a class to implement the TableReader interface

import java.io.File;
import java.io.FileReader;
import java.io.Reader;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.ArrayList;
import java.util.List;

import org.relique.io.TableReader;
import org.relique.jdbc.csv.CsvConnection;
import org.relique.jdbc.csv.CsvDriver;

public class CSVQuery implements TableReader{

    public static void main(String[] args) throws Exception {
        CSVQuery q = new CSVQuery();
       
        Reader reader = new FileReader(
                new File("/path/to/some/file.csv"));

        q.executeQuery(reader);
       
        reader.close();
    }

    public void executeQuery(Reader reader) throws Exception{

        Class.forName("org.relique.jdbc.csv.CsvDriver");
        CsvConnection conn = (CsvConnection)DriverManager.getConnection("jdbc:relique:csv:class:CSVQuery");

        conn.setReader(reader);       
        Statement stmt = conn.createStatement();
        ResultSet rs = stmt.executeQuery("SELECT * FROM myTable where increment like 'FTP%'");
        CsvDriver.writeToCsv(rs, System.out, true);

        rs.close();
        stmt.close();
        conn.close();
    }

    @Override
    public Reader getReader(Statement stmt, String table) throws SQLException {
        CsvConnection csvConn = (CsvConnection)stmt.getConnection();
        return csvConn.getReader();
    }

    @Override
    public List<String> getTableNames(Connection conn) throws SQLException {
        ArrayList<String> names = new ArrayList<String>();
        names.add("myTable");
        return names;
    }
}


and that's it.


Saturday, August 24, 2013

Next, the frontend, using primefaces mobile

Ok, now that we have the backend, it's time to start playing with some frontend. My first idea was to create something I could access from the blue-ray player or from the smart tv, but neither have a decent browser. Then I've considered learning a little android app development, but the UI is not incredibly complex.

Then I remembered Primeface Mobile UI. It's just JSF, so it's fast and easy. And I've tried a little and it looks good in my xperia :-) It's powered by jQuery mobile, so it's good stuff IMO.

Things that weren't obvious for those who does not like to wait, about primefaces mobile configuration:

[1] you have to install both primefaces and primefaces mobile jars, since the renderer is in the "core" primefaces jar.

<f:view xmlns="http://www.w3.org/1999/xhtml"
        xmlns:f="http://java.sun.com/jsf/core"
        xmlns:h="http://java.sun.com/jsf/html"
        xmlns:ui="http://java.sun.com/jsf/facelets"
        xmlns:p="http://primefaces.org/ui"
        xmlns:pm="http://primefaces.org/mobile"       
        contentType="text/html"
        renderKitId="PRIMEFACES_MOBILE">

    <pm:page title="Components">
        <pm:view id="main">
            <pm:header title="Header" />
            <pm:content>Content</pm:content>
        </pm:view>
    </pm:page>
</f:view>


[2] the minimal web.xml you need (I like xhtml)

<!DOCTYPE web-app PUBLIC
 "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
 "http://java.sun.com/dtd/web-app_2_3.dtd" >

<web-app>
    <display-name>Movies</display-name>
    <context-param>
        <param-name>com.sun.faces.allowTextChildren</param-name>
        <param-value>true</param-value>
    </context-param>
    <servlet>
        <servlet-name>Faces Servlet</servlet-name>
        <servlet-class>javax.faces.webapp.FacesServlet</servlet-class>
    </servlet>
    <servlet-mapping>
        <servlet-name>Faces Servlet</servlet-name>
        <url-pattern>*.xhtml</url-pattern>
    </servlet-mapping>
    <welcome-file-list>
        <welcome-file>/index.xhtml</welcome-file>
    </welcome-file-list>
</web-app>



Friday, August 23, 2013

How to add the torrent to your local uTorrent via Web API

after getting the torrent (see http://leonotepad.blogspot.com.br/2013/08/how-to-find-torrent-for-legend-youve.html), now it's time to download it.

uTorrent has some nice web API. This means that if you're running uTorrent at your machine, you can also control it using some http requests.

so, first, enable the web UI.

then, you'll need the (new) apache http client (or just anything that can deal with http BASIC auth)

please notice that you have to get the token. (don't worry, see the example below).

so now, just put the pieces together and make your own movie downloader at home :-)

package movies.backend;

import java.io.InputStream;
import java.io.StringWriter;
import java.net.URLEncoder;

import org.apache.commons.io.IOUtils;
import org.apache.http.HttpEntity;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.AuthCache;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.protocol.ClientContext;
import org.apache.http.impl.auth.BasicScheme;
import org.apache.http.impl.client.BasicAuthCache;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.protocol.BasicHttpContext;
import org.apache.http.util.EntityUtils;

public class TorrentDownload {

    public static void main(String[] args) throws Exception {

        HttpHost targetHost = new HttpHost("127.0.0.1", 8888, "http");

        DefaultHttpClient httpclient = new DefaultHttpClient();
        try {
            httpclient.getCredentialsProvider().setCredentials(
                    new AuthScope(targetHost.getHostName(), targetHost.getPort()),
                    new UsernamePasswordCredentials("xxx", "xxx"));

            // Create AuthCache instance
            AuthCache authCache = new BasicAuthCache();
            // Generate BASIC scheme object and add it to the local
            // auth cache
            BasicScheme basicAuth = new BasicScheme();
            authCache.put(targetHost, basicAuth);

            // Add AuthCache to the execution context
            BasicHttpContext localcontext = new BasicHttpContext();
            localcontext.setAttribute(ClientContext.AUTH_CACHE, authCache);

            HttpGet httpget = new HttpGet("http://127.0.0.1:8888/gui/");
            HttpResponse response = httpclient.execute(targetHost, httpget, localcontext);
            EntityUtils.consumeQuietly(response.getEntity());
           
            httpget = new HttpGet("http://127.0.0.1:8888/gui/token.html");
            response = httpclient.execute(targetHost, httpget, localcontext);
           
            HttpEntity e = response.getEntity();
            InputStream is = e.getContent();
            StringWriter sw = new StringWriter();
            IOUtils.copy(is, sw);
            sw.flush();
            sw.close();
            is.close();
            //<html><div id='token' style='display:none;'>gzB9zbMru3JJlBf2TbmwwklESgXW2hD_caJfFLvNBjmaRbLZ3kNGnSHrFlIAAAAA</div></html>
            String t = sw.toString();
            int start = "<html><div id='token' style='display:none;'>".length();
            int end = t.indexOf("</div></html>");
            String token = t.substring(start,end);
            System.out.println(token);
            EntityUtils.consumeQuietly(response.getEntity());
           
            String add = URLEncoder.encode("http://www.torrentportal.com/download/6066218/True.Blood.S06E01.Who.Are.You.Really.XviD-MGD%5Bettv%5D.torrent","UTF-8");
            httpget = new HttpGet("http://127.0.0.1:8888/gui/?action=add-url&s="+add+"&token="+token);
            response = httpclient.execute(targetHost, httpget, localcontext);
           
            e = response.getEntity();
            is = e.getContent();
            sw = new StringWriter();
            IOUtils.copy(is, sw);
            sw.flush();
            sw.close();
            is.close();
            System.out.println(sw.toString());

        } finally {
            // When HttpClient instance is no longer needed,
            // shut down the connection manager to ensure
            // immediate deallocation of all system resources
            httpclient.getConnectionManager().shutdown();
        }
    }
}


How to find the torrent for the legend you've got at opensubtitles.org using Isohunt web API

after getting the subtitle (see http://leonotepad.blogspot.com.br/2013/08/how-to-search-and-retrieve-subtitle.html), now it's time to find the torrent.

Isohunt has some nice web API.

package movies.backend;

import java.io.IOException;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLEncoder;
import java.util.List;
import java.util.Map;

import flexjson.JSONDeserializer;

public class IsohuntTorrentSearch {

    /**
     * @param args
     * @throws IOException
     */
    public static void main(String[] args) throws IOException {
        URL url = new URL("http://ca.isohunt.com/js/json.php?ihq="+URLEncoder.encode("True.Blood.S06E01.Who.Are.You.Really.720p","UTF-8")+"&start=21&rows=20&sort=seeds");
        Map json = (Map)new JSONDeserializer().deserialize(new InputStreamReader(url.openStream()));
        Map items = (Map)json.get("items");
        List list = (List)items.get("list");
        System.out.println(list.get(0));
        Map item = (Map)list.get(0);
        System.out.println(item.get("tracker_url"));
    }
}

How to search and retrieve a subtitle from opensubtitles.org using Java

First, you'll have to ask them for a valid user agent, but for test purposes, you can use "OS Test User Agent".

All the API details are described at http://trac.opensubtitles.org/projects/opensubtitles/wiki/XMLRPC.

You'll need

commons-io-2.4.jar
commons-logging-1.1.jar
ws-commons-util-1.0.2.jar
xmlrpc-client-3.1.3.jar
xmlrpc-common-3.1.3.jar


and Oracle java 7.

package movies.backend;

import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.StringWriter;
import java.net.URL;
import java.util.HashMap;
import java.util.Map;
import java.util.zip.GZIPInputStream;

import org.apache.commons.io.IOUtils;
import org.apache.xmlrpc.XmlRpcException;
import org.apache.xmlrpc.client.XmlRpcClient;
import org.apache.xmlrpc.client.XmlRpcClientConfigImpl;

import sun.misc.BASE64Decoder;

public class SubtitleSearchService {

    /**
     * @param args
     * @throws XmlRpcException
     * @throws IOException
     */
    public static void main(String[] args) throws XmlRpcException, IOException {
        XmlRpcClientConfigImpl config = new XmlRpcClientConfigImpl();
        config.setServerURL(new URL("http://api.opensubtitles.org/xml-rpc"));
        XmlRpcClient client = new XmlRpcClient();
        client.setConfig(config);
        Object username = "xxx";
        Object password = "xxx";
        Object language = "pob";
        Object useragent = "OS Test User Agent";
        Object[] params = new Object[] { username, password, language, useragent };
        Map result = (Map) client.execute("LogIn", params);
        String status = (String) result.get("status");
        String token = (String) result.get("token");

        Object imdbQuery = "true blood";
        params = new Object[] { token, imdbQuery  };
        result = (Map) client.execute("SearchMoviesOnIMDB", params);
       
        System.out.println(result);
       
        Object[] data = (Object[])result.get("data");
        Map dataMap = (Map)data[0];
        Object id = dataMap.get("id");
        Object title = dataMap.get("title");
       
        Map movieQuery = new HashMap();
        movieQuery.put("sublanguageid", "pob");
        movieQuery.put("imdbid", id);
       
        Object[] listQuery = {movieQuery};
       
        params = new Object[] { token, listQuery   };
        result = (Map) client.execute("SearchSubtitles", params);
       
        System.out.println(result);
       
        data = (Object[])result.get("data");
        dataMap = (Map)data[0];
       
        System.out.println(dataMap.get("SubFileName"));
        //SubDownloadLink=http://dl.opensubtitles.org/en/download/filead/1953590867.gz/sid-ojl76mmpi89o8tvpb3ebl45d83
        System.out.println(dataMap.get("SubDownloadLink"));
       
        String downloadLink = (String)dataMap.get("SubDownloadLink");
        downloadLink = downloadLink.replaceFirst("http://dl.opensubtitles.org/en/download/filead/", "");
        Object subFileId = downloadLink.substring(0, downloadLink.indexOf(".gz"));

        Map listOfSubtitles = new HashMap();
        listOfSubtitles.put("data", subFileId);
        params = new Object[] { token, listOfSubtitles    };
        result = (Map) client.execute("DownloadSubtitles", params);
       
        System.out.println(result);
       
        data = (Object[])result.get("data");
        dataMap = (Map)data[0];
       
        System.out.print(dataMap.get("data"));
       
        BASE64Decoder decoder = new BASE64Decoder();
        byte[] gzip = decoder.decodeBuffer((String)dataMap.get("data"));

        GZIPInputStream gis = new GZIPInputStream(new ByteArrayInputStream(gzip));
        StringWriter sw = new StringWriter();
       
        IOUtils.copy(gis, sw);
        sw.flush();
        sw.close();
        gis.close();
       
        System.out.println(sw.toString());
       
        params = new Object[] { token };
        client.execute("LogOut", params);
       
    }

}


enjoy

Thursday, July 18, 2013

behave yourself

useful for sysadmins

http://sdubois.evolix.net/blog/2011/09/08/lshell-an-alternative-to-ssh-chroot-for-your-users/

lshell

very very useful to restrict your users

Wednesday, July 3, 2013

Flexjson vs GSON

GSON is prettier, but flexjson deals with cycles.

So I usually go flexjson. It's not pretty to die by stack overflow.

Booleans in OpenJPA

try

    @ExternalValues({"true=1", "false=0"})
    @Type(int.class)
    boolean val;


or MySQL will use a single bit to store this

So you want a mail server to play with, try apache james

Forget sendmail. Download apache james, and

1. vi bin/phoenix.sh and add a JAVA_HOME pointing to some non-IBM JVM such as sun's

#   JAVA_HOME          Must point at your Java Development Kit installation.
#export JAVA_HOME=/home/leoks/ibm-java-i386-70/
export JAVA_HOME=/opt/j2sdk1.4.2_19/


just because IBM JVM does not have the encryption algorithm you'll need.

2. vi apps/james/SAR-INF/config.xml and change all the ports to something higher, so you don't have to be root to start it (basically POP3=9110, NNTP=9119 and SMTP=9025)

3. start it up

[leoks@xxx bin]$ ./run.sh
Using PHOENIX_HOME:   /home/leoks/Downloads/james-2.3.2
Using PHOENIX_TMPDIR: /home/leoks/Downloads/james-2.3.2/temp
Using JAVA_HOME:      /opt/j2sdk1.4.2_19/
Running Phoenix:

Phoenix 4.2

James Mail Server 2.3.2
Remote Manager Service started plain:4555
POP3 Service started plain:9110
SMTP Service started plain:9025
NNTP Service started plain:9119
FetchMail Disabled

^Z
[1]+  Stopped                 ./run.sh
[leoks@xxx bin]$ bg


4. then add your users (user and password are root/root)

[leoks@xxx bin]$ telnet localhost 4555
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
JAMES Remote Administration Tool 2.3.2
Please enter your login and password
Login id:
root
Password:
root
Welcome root. HELP for a list of commands
adduser leoks xxx
User leoks added
quit
Bye
Connection closed by foreign host.


and you're good to go

Wednesday, June 5, 2013

The problem(s) with EJB @Scheduler

Really annoyed with the new EJB 3.1 @Scheduler API implementation. Not only because of JSR, which lacks of basic functionality such as defining where to persist jobs and how to manage them, but with the implementations.

The JSR is crystal clear when it states

"Timers are persistent. If the server is shut down (or even crashes), timers are saved and will become active again when the server is restarted. If a timer expires while the server is down, the container will call ejbTimeout when the server is restarted." (JEE tutorial)

problem #1

JSR does not want to improve it. Really. Check this out. Come on.

problem #2


Then, JBOSS implementation was clearing the jobs after server shutdown.

problem #3

Then TomEE+ implementation promises to properly implement the jobs persistence in the 1.6.0... well, not really. 

So, are the JEE containers really taking the specs seriously?

:-(

Going back to Quartz.

And sorry for the ignorance, but reading http://docs.oracle.com/javaee/6/tutorial/doc/bnboy.html , tell me, what part of the code is expected to be executed when a programmatic timer scheduled reaches the timeout?

Tuesday, June 4, 2013

ActiveMQ MySQL Tables

haven't found anywhere, so I had to download the damn activeMQ and run it in order to get these tables.

so I've edited activemq.xml and changed

        <persistenceAdapter>
        <jdbcPersistenceAdapter dataSource="#myds"/>
        </persistenceAdapter>


and added the bean after

    (...)<import resource="jetty.xml"/>

<bean id="myds" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
    <property name="driverClassName" value="com.mysql.jdbc.Driver"/>
    <property name="url" value="jdbc:mysql://localhost/activemq?relaxAutoCommit=true"/>
    <property name="username" value="xxx"/>
    <property name="password" value="xxx"/>
    <property name="poolPreparedStatements" value="true"/>
</bean>


nasty how I have to reference one id to another using this hash (#) thing...

so that's what I've got

mysql> show tables;
+--------------------+
| Tables_in_activemq |
+--------------------+
| ACTIVEMQ_ACKS      |
| ACTIVEMQ_LOCK      |
| ACTIVEMQ_MSGS      |
+--------------------+
3 rows in set (0.00 sec)


and finally

mysql> show create table ACTIVEMQ_ACKS;
+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table         | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                 |
+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ACTIVEMQ_ACKS | CREATE TABLE `ACTIVEMQ_ACKS` (
  `CONTAINER` varchar(250) NOT NULL,
  `SUB_DEST` varchar(250) DEFAULT NULL,
  `CLIENT_ID` varchar(250) NOT NULL,
  `SUB_NAME` varchar(250) NOT NULL,
  `SELECTOR` varchar(250) DEFAULT NULL,
  `LAST_ACKED_ID` bigint(20) DEFAULT NULL,
  `PRIORITY` bigint(20) NOT NULL DEFAULT '5',
  `XID` longblob,
  PRIMARY KEY (`CONTAINER`,`CLIENT_ID`,`SUB_NAME`,`PRIORITY`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 |
+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.06 sec)

mysql> show create table ACTIVEMQ_LOCK;
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table         | Create Table                                                                                                                                                                                        |
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ACTIVEMQ_LOCK | CREATE TABLE `ACTIVEMQ_LOCK` (
  `ID` bigint(20) NOT NULL,
  `TIME` bigint(20) DEFAULT NULL,
  `BROKER_NAME` varchar(250) DEFAULT NULL,
  PRIMARY KEY (`ID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 |
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql> show create table ACTIVEMQ_MSGS;
+---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table         | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
+---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ACTIVEMQ_MSGS | CREATE TABLE `ACTIVEMQ_MSGS` (
  `ID` bigint(20) NOT NULL,
  `CONTAINER` varchar(250) DEFAULT NULL,
  `MSGID_PROD` varchar(250) DEFAULT NULL,
  `MSGID_SEQ` bigint(20) DEFAULT NULL,
  `EXPIRATION` bigint(20) DEFAULT NULL,
  `MSG` longblob,
  `PRIORITY` bigint(20) DEFAULT NULL,
  `XID` longblob,
  PRIMARY KEY (`ID`),
  KEY `ACTIVEMQ_MSGS_MIDX` (`MSGID_PROD`,`MSGID_SEQ`),
  KEY `ACTIVEMQ_MSGS_CIDX` (`CONTAINER`),
  KEY `ACTIVEMQ_MSGS_EIDX` (`EXPIRATION`),
  KEY `ACTIVEMQ_MSGS_PIDX` (`PRIORITY`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 |
+---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)