Java IoT Authors: Yeshim Deniz, Liz McMillan, Mano Marks, Elizabeth White, Pat Romanski

Related Topics: Java IoT

Java IoT: Article

Managing a Standardized Build Process Outside of the Eclipse IDE

Point-and-click solutions won't cut it

Building objects in the Eclipse IDE is simple - it's a point-and-click solution. However, as applications built on the Eclipse platform mature the need for building outside of the IDE increases. This need can be driven by the development team that is striving to perform agile development techniques where builds are executed based on a file "check-in" action into an SCM tool. The need can also be driven by IT governance where a scheduled and audited production build is required. Moving from builds managed inside of the Eclipse platform to builds managed outside of the Eclipse platform can be a big task in itself. Don't hesitate to make this jump. It's a jump that you'll find you can't do without. The sooner you get out of your point-and-click build process, the sooner your application will begin to mature.

Defining the build process should not be taken lightly. Auditability and traceability of built objects are becoming increasingly important due to IT compliance mandates. This means that your builds must become more traceable than a point-and-click process. Don't make the mistake of addressing the issue of building outside of the Eclipse IDE long after the application has grown to an unmanageable size. Delaying the inevitable only results in a poorly managed, unplanned, ad hoc build process that isn't sustainable, can't meet IT compliance, and involves expensive hidden costs in maintenance and fixes.

There are three ways of addressing the build process outside the Eclipse IDE. The most common method is to manually develop and maintain Ant/XML scripts. These scripts use Ant Tasks from the Apache Foundation to act as a wrapper to the Java compiler. The second method is to write scripts that call Eclipse in what's called a "headless" mode. A script that executes the build in headless mode acts in the same way as the point-and-click process inside the Eclipse IDE, but does the build from a script. And finally, the preferred method is to use a commercial build tool that can automate the creation of the scripts. Commercial tools that minimize the use of manual scripts establish a more repeatable and traceable build workflow, the ultimate goal of any solid development process.

If you don't have the luxury of a commercial build tool that can create a solid reusable build framework, you must create a manual build process that's as standardized as possible. A "manual" build process refers to any scripted build process that has to be maintained manually. Even if you execute your manual scripts through a job scheduling build management tool, your builds are still manual because you must manually maintain the build logic contained in the scripts.

When writing the manual build process, your choices become writing Ant/XML scripts to perform the build or to use the Eclipse headless mode option. Establishing a repeatable build that can be traced requires that you concisely define how the build executes and with what source code. The use of the headless mode removes that level of control. It provides a little more functionality than using the point-and-click process inside Eclipse. When running a headless mode script, you're still relying on the Eclipse IDE to control the build. For this reason, defining a build process using Ant/XML is recommended if a commercial tool isn't available.

If you review any Ant/XML script, it may appear that the process of converting Java source into Java jars is complicated. This isn't necessarily true. Ant/XML files execute in a serial fashion, top to bottom. Everything must be precisely coded in a particular order. That's why XML build scripts can be very large and difficult to debug. There are some suggested standards on writing XML scripts, but they're not always followed. At a minimum, XML build scripts should follow a basic flow with pre-processing and post-processing steps that are consistent for every XML script created, planned, or unplanned.

Pre-Processing Steps
Pre-processing steps are used to establish the environment in which the subsequent task will execute. The point of setting up these pre-processing steps is to get the source code and variables organized and do overall housekeeping before compilation starts. A common mistake is to mix together these housekeeping steps for each call to the Java compiler or before calling an Ant Task. By organizing the pre-processing steps at the beginning of the XML scripts, the build process becomes clearer and easier to follow. It also reduces redundancy and results in an improved, more efficient process. These are the recommended pre-processing steps:

The CLASSPATH identifies which Java Classes are going to be used to resolve inter-class dependencies. The Java compiler will search the CLASSPATH in a first-found method and use a file as soon as it's been located. The CLASSPATH can include Jar and Zip files and directories.

When setting the CLASSPATH it's important to make sure of two things. First, the CLASSPATH should only be defined once in your process. Second, only the jar files and class directories that are used should be referenced in the CLASSPATH. Don't reference other unused jars since then the Java compiler will do more work than needed, slowing down your build substantially. And specifying only the used jars will make for quick dependency identification. There's nothing worse than attempting to trace jar file dependencies only to find that a large number of jar files aren't needed. This can add substantial time to debugging your builds.

The use of wild cards is always a topic of debate. Wild cards can eliminate typing in your script, but in the end may cause your build process to include more objects than needed. List each jar file in the CLASSPATH explicitly to prevent any vagueness. This is particularly critical when exposing your build details for IT compliance mandates. Wild cards aren't traceable.

Setting the CLASSPATH should be the first task in the Ant XML script.

Copying and Renaming Files
There are cases in which source code, jar and property files have to be copied from one location to another for the compiler to find the file or put it in the archive correctly. Do yourself a favor and minimize the use of copying and renaming of files. It makes it extremely difficult to trace the archive contents back to the original source. Copying files around also creates a more "magical" build process. IT compliance mandates want a clear view into your process. No "magic" is needed.

Instead of copying or renaming the files, put the files in the correct location from the start. Don't use your Ant/XML script to clean up a mistake in file organization. This may involve updating your project directory structure and making your Source Code Management tool more efficient. As an alternative to copying and renaming files, the use of the Ant Task "Zip" and its attributes, such as "dir" and "prefix" can handle getting source from one location and putting it in the archive at a different location.

This sample XML code from the Apache Ant Manual demonstrates using the Ant Task "Zip" to take one source location (htdocs/manual) and put it in another location (docs/user-guide):

<zip destfile="${dist}/manual.zip">
     <zipfileset dir="htdocs/manual" prefix="docs/user-guide"/>
     <zipgroupfileset dir="." includes="examples*.zip"/>

Compile Step
The compile step is, of course, the heart of your process. It will become the largest section of XML script. The important point to remember when defining this portion of your script is the management of dependencies.

Dependency References
With Ant, you can explicitly define the dependencies between tasks. For example, the JAR task can be dependent on the JAVAC task. Ant will also let multiple task dependencies be established. Don't be seduced by this seemingly convenient Ant Task. While it seems useful, it can be burdensome. When tracing the order of executing the various Ant tasks in the Ant/XML script, it's much easier to follow a dependency chain that has only one task dependency instead of multiple ones. For example:

Scenario 1
JAR Task depends on JAVAC Task
JAVAC Task depends on the COPY Task
COPY Task depends on the INITIALIZATION Task

Scenario 2
JAR Task depends on the JAVAC Task and COPY Task
JAVAC Task depends on the COPY Task and INITIALIZATION Task
COPY Task depends on the INITIALIZATION Task

As you can see in Scenario 2, there are redundant Ant task dependencies. For example, the COPY Task is redundant on the JAR Task. This redundant use of COPY Tasks isn't needed since it's already referenced higher up in the task dependency hierarchy being the JAVAC Task.

There will be cases when you want to have multiple task dependencies as in the creation of a war file. In this case, multiple task dependencies may be needed to ensure that all of the jars are created before the war. But each jar should have just one task dependency, that being the JAVAC task.

Identifying Source Code
Finding your source can seem like an easy item at first, but when applications get bigger there's a greater chance that wrong or obsolete code is included. Using wildcards in the Ant/XML script is an easy way to minimize the need for typing, but for the wildcards to be effective, the source files have to be efficiently organized in a proper Java package directory structure.

The best way to manage source is to define an efficient package directory structure. So you must move beyond your unique needs and address the package names at a more global level in your organization. It's best to make sure that a corporate Java package structure is agreed on and used. As part of the Java package structure it's best to keep the package names simple. Really long package names can cause problems with the file limits on the Windows operating system. Java compiles on Windows have been known to stop working when the 254-character limit is exceeded. To make this problem even peskier, the script may work on one user's machine but break on another's. This is due do the build directory root name being added to the package names. For instance, one person may do the build in c:\mybuilds but another may build in d:\onlinedata\j2ee\development\code. The difference in the directory name can make or break the build by pushing the 254-character limit.

Another item that defines the location of source code is the use of the excludes attribute of the JAVAC Ant Task. It's best to remove the older obsolete code from the Source Code Management tool and from the file system instead of using the exclude attribute. Most SCM tools provide for renaming or removing an item without loosing all of the history. SCM tools also allow for comments that create a level of traceability on why a piece of code is no longer required. Having this information in the SCM tool makes for easier access versus the information being hidden in a comment in the Ant XML.

JAVAC -sourcepath
The native command line Java compiler (javac.exe) has an interesting flag called -sourcepath that provides a directory concatenation to find the source code. It works on a first-found basis. So once the source has been located, the directory browse stops. There are two advantages to using this parameter. First, all of the code doesn't have to be found in the current build directory. That is, source code can be found in multiple locations. Thus, the build process only has to check out the changed code and find the remaining code from a previous full checkout. This will speed the overall build process by minimizing the files to be checked out. Second, if the JAVAC command is given a Java file as a parameter it will then check using the -sourcepath parameter for additional source that's referenced by the original Java file and compile it too. This process will allow just the changed source to be passed as the parameters to JAVAC and JAVAC will figure out all the remaining dependencies for you.

More Stories By Steve Taylor

Steve Taylor is an experienced senior developer, bringing 17 years of expertise with client/server and mainframe application development and system integration. Prior to founding Catalyst Systems Corporation, he served as a lead technical consultant responsible for the successful implementation of applications into the production environment. Steve received his BS in computer science/mathematics from the University of Illinois-CU.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

@ThingsExpo Stories
Bert Loomis was a visionary. This general session will highlight how Bert Loomis and people like him inspire us to build great things with small inventions. In their general session at 19th Cloud Expo, Harold Hannon, Architect at IBM Bluemix, and Michael O'Neill, Strategic Business Development at Nvidia, discussed the accelerating pace of AI development and how IBM Cloud and NVIDIA are partnering to bring AI capabilities to "every day," on-demand. They also reviewed two "free infrastructure" pr...
In his keynote at @ThingsExpo, Chris Matthieu, Director of IoT Engineering at Citrix and co-founder and CTO of Octoblu, focused on building an IoT platform and company. He provided a behind-the-scenes look at Octoblu’s platform, business, and pivots along the way (including the Citrix acquisition of Octoblu).
Data is an unusual currency; it is not restricted by the same transactional limitations as money or people. In fact, the more that you leverage your data across multiple business use cases, the more valuable it becomes to the organization. And the same can be said about the organization’s analytics. In his session at 19th Cloud Expo, Bill Schmarzo, CTO for the Big Data Practice at Dell EMC, introduced a methodology for capturing, enriching and sharing data (and analytics) across the organization...
The explosion of new web/cloud/IoT-based applications and the data they generate are transforming our world right before our eyes. In this rush to adopt these new technologies, organizations are often ignoring fundamental questions concerning who owns the data and failing to ask for permission to conduct invasive surveillance of their customers. Organizations that are not transparent about how their systems gather data telemetry without offering shared data ownership risk product rejection, regu...
Grape Up is a software company, specialized in cloud native application development and professional services related to Cloud Foundry PaaS. With five expert teams that operate in various sectors of the market across the USA and Europe, we work with a variety of customers from emerging startups to Fortune 1000 companies.
Financial Technology has become a topic of intense interest throughout the cloud developer and enterprise IT communities. Accordingly, attendees at the upcoming 20th Cloud Expo at the Javits Center in New York, June 6-8, 2017, will find fresh new content in a new track called FinTech.
SYS-CON Events announced today that Interoute, owner-operator of one of Europe's largest networks and a global cloud services platform, has been named “Bronze Sponsor” of SYS-CON's 20th Cloud Expo, which will take place on June 6-8, 2017 at the Javits Center in New York, New York. Interoute is the owner-operator of one of Europe's largest networks and a global cloud services platform which encompasses 12 data centers, 14 virtual data centers and 31 colocation centers, with connections to 195 add...
Multiple data types are pouring into IoT deployments. Data is coming in small packages as well as enormous files and data streams of many sizes. Widespread use of mobile devices adds to the total. In this power panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists will look at the tools and environments that are being put to use in IoT deployments, as well as the team skills a modern enterprise IT shop needs to keep things running, get a handle on all this data, and deli...
The age of Digital Disruption is evolving into the next era – Digital Cohesion, an age in which applications securely self-assemble and deliver predictive services that continuously adapt to user behavior. Information from devices, sensors and applications around us will drive services seamlessly across mobile and fixed devices/infrastructure. This evolution is happening now in software defined services and secure networking. Four key drivers – Performance, Economics, Interoperability and Trust ...
The Internet of Things is clearly many things: data collection and analytics, wearables, Smart Grids and Smart Cities, the Industrial Internet, and more. Cool platforms like Arduino, Raspberry Pi, Intel's Galileo and Edison, and a diverse world of sensors are making the IoT a great toy box for developers in all these areas. In this Power Panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists discussed what things are the most important, which will have the most profound e...
@ThingsExpo has been named the Most Influential ‘Smart Cities - IIoT' Account and @BigDataExpo has been named fourteenth by Right Relevance (RR), which provides curated information and intelligence on approximately 50,000 topics. In addition, Right Relevance provides an Insights offering that combines the above Topics and Influencers information with real time conversations to provide actionable intelligence with visualizations to enable decision making. The Insights service is applicable to eve...
SYS-CON Events announced today that Grape Up will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct. 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Grape Up is a software company specializing in cloud native application development and professional services related to Cloud Foundry PaaS. With five expert teams that operate in various sectors of the market across the U.S. and Europe, Grape Up works with a variety of customers from emergi...
SYS-CON Events announced today that Hitachi, the leading provider the Internet of Things and Digital Transformation, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Hitachi Data Systems, a wholly owned subsidiary of Hitachi, Ltd., offers an integrated portfolio of services and solutions that enable digital transformation through enhanced data management, governance, mobility and analytics. We help globa...
SYS-CON Events announced today that SoftLayer, an IBM Company, has been named “Gold Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2016, at the Javits Center in New York, New York. SoftLayer, an IBM Company, provides cloud infrastructure as a service from a growing number of data centers and network points of presence around the world. SoftLayer’s customers range from Web startups to global enterprises.
20th Cloud Expo, taking place June 6-8, 2017, at the Javits Center in New York City, NY, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy.
SYS-CON Events announced today that Super Micro Computer, Inc., a global leader in compute, storage and networking technologies, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Supermicro (NASDAQ: SMCI), the leading innovator in high-performance, high-efficiency server technology, is a premier provider of advanced server Building Block Solutions® for Data Center, Cloud Computing, Enterprise IT, Hadoop/...
Amazon has gradually rolled out parts of its IoT offerings in the last year, but these are just the tip of the iceberg. In addition to optimizing their back-end AWS offerings, Amazon is laying the ground work to be a major force in IoT – especially in the connected home and office. Amazon is extending its reach by building on its dominant Cloud IoT platform, its Dash Button strategy, recently announced Replenishment Services, the Echo/Alexa voice recognition control platform, the 6-7 strategic...
Judith Hurwitz is president and CEO of Hurwitz & Associates, a Needham, Mass., research and consulting firm focused on emerging technology, including big data, cognitive computing and governance. She is co-author of the book Cognitive Computing and Big Data Analytics, published in 2015. Her Cloud Expo session, "What Is the Business Imperative for Cognitive Computing?" is scheduled for Wednesday, June 8, at 8:40 a.m. In it, she puts cognitive computing into perspective with its value to the busin...
Cognitive Computing is becoming the foundation for a new generation of solutions that have the potential to transform business. Unlike traditional approaches to building solutions, a cognitive computing approach allows the data to help determine the way applications are designed. This contrasts with conventional software development that begins with defining logic based on the current way a business operates. In her session at 18th Cloud Expo, Judith S. Hurwitz, President and CEO of Hurwitz & ...
Cybersecurity is a critical component of software development in many industries including medical devices. However, code is not always written to be robust or secure from the unknown or the unexpected. This gap can make medical devices susceptible to cybersecurity attacks ranging from compromised personal health information to life-sustaining treatment. In his session at @ThingsExpo, Clark Fortney, Software Engineer at Battelle, will discuss how programming oversight using key methods can incre...