开源的Java ETL (Extraction, Transform, Load) 工具
ETL,是英文 Extract-Transform-Load 的缩写,用来描述将数据从来源端经过萃取(extract)、转置(transform)、加载(load)至目的端的过程。ETL一词较常用在数据仓库,但其对象并不限于数据仓库。
ETL所描述的过程,一般常见的作法包含ETL或是ELT(Extract-Load-Transform),并且混合使用。通常愈大量的数据、复杂的转换逻辑、目的端为较强运算能力的数据库,愈偏向使用ELT,以便运用目的端数据库的平行处理能力。
ETL(or ELT)的流程可以用任何的编程语言去开发完成,由于ETL是极为复杂的过程,而手写程序不易管理,有愈来愈多的企业采用工具协助ETL的开发,并运用其内置的metadata功能来储存来源与目的的对应(mapping)以及转换规则。
工具并可以提供较强大的连接功能(connectivity)来连接来源及目的端,开发人员不用去熟悉各种相异的平台及数据的结构,亦能进行开发。
下面是一些开源的Java ETL工具
- Octopus
- Octopus is a simple Java-based Extraction, Transform, and Loading (ETL) tool. It may connect to any JDBC data sources and perform transformations defined in an XML file. A loadjob-generator is provided to generate Octopus loadjob skeletons from an existing database. Many different types of databases can be mixed (MSSQL, Oracle, DB2, QED, JDBC-ODBC with Excel and Access, MySQL, CSV-files, XML-files,...) Three special JDBC drivers come with Octopus to support JDBC access to CSV-files (CSV-JDBC), MS-SQL (FreeTDS) and XML. Octopus supports Ant and JUnit to create a database / tables and extract/load data during a build or test process.
- Xineo
- Xineo XIL (XML Import Langage) defines an XML language for transforming various record-based data sources into XML documents, and provides a fully functional XIL processing implementation. This implementation has built-in support for relational (via JDBC) and structured text (like CSV) sources, and is extensible thanks to its public API, allowing dynamic integration of new data source implementations. It also provides an abstraction over output format, and the Xineo implementation can generate output documents into stream or as DOM document. Xineo's implementation built-in data sources include : Relational data via JDBC and Structured text via regular expressions.
- CloverETL
- CloverETL Features include internally represents all characters as 16bit, converts from most common character sets (ASCII, UTF-8, ISO-8859-1,ISO-8859-2, etc), works with delimited or fix-length data records, data records (fields) are internally handled as a variable-length data structures, fields can have default values, handles NULL values, cooperates with any database with JDBC driver, transforming of the data is performed by independent components, each running as an independent thread, framework implements so called pipeline-parallelism, metadata describing structure of data files (records) can be read from XML and transformation graphs can be read from XML
- BabelDoc
- BabelDoc is a Java framework for processing documents in linear stages, it tracks documents and can reintroduce documents back into into the pipelines, it is monitorable and configurable through a number of interfaces, it can be run standalone, in server processes or in application servers, it can be reconfigured dynamically by text files and database tables.
- Joost
- Java implementation of the Streaming Transformations for XML (STX) language. Streaming Transformations for XML (STX) is a one-pass transformation language for XML documents. STX is intended as a high-speed, low memory consumption alternative to XSLT. Since it does not require the construction of an in-memory tree, it is suitable for use in resource constrained scenarios.
- CB2XML
- CB2XML (CopyBook to XML) is a COBOL CopyBook to XML converter written in Java and based on the SableCC parser generator. This project includes utilities to convert an XML instance file into its COBOL copybook equivalent string buffer and vice versa. You can find additional information about supporting Jurasic systems here
.
- mec-eagle
- JAVA XML XSL B2B integration software:SWING based GUI,an EDI to XML, XML to XML and XML to EDI converter,client-server architecture.All EDI standards are supported:EDIFACT,ANSI X.12,SAP IDOC,XCBL,RosettaNet,Biztalk.Included comm:SMTP,FTP,HTTP(S),PGP/MIME
- Transmorpher
- Transmorpher is an environment for processing generic transformations on XML documents. It aims at complementing XSLT in order to:
- describe easily simple transformations (removing elements, replacing tag and attribute names, concatenating documents...);
- allowing regular expression transformations on the content;
- composing transformations by linking their (multiple) output to input;
- iterating transformations, sometimes until saturation (closure operation);
- integrating external transformations.
- XPipe
- XPipe is an approach to manageable, scaleable, robust XML processing based on the assembly line principle, common in many areas of manufacturing. XPipe as being an attempt to take what was great about the original Unix pipe idea and apply it for structured information streams based on XML.
- DataSift
- DataSift is a powerful java data validation and transformation framework, aimed at enterprise software development, which provides developers with an extensible architecture they can fully adapt. Almost every feature in it can be configured and extended in some way.
- Xephyrus Flume
- Flume is a component pipeline engine. It allows you to chain together multiple workers into a pipeline mechanism. The intention of Flume is that each of the workers would provide access to a different type of technology. For example, a pipeline could consist of a Jython script worker followed by a BeanShell script worker followed by an XSLT worker.
- Smallx - Smallx supports streaming of XML infosets to allow processing of very large documents (500MB-1GB). Processing is specified in an XML syntax that describes an XML pipeline--which is a sequence of components that consume and produce infosets. This allows chaining of XML component standards like XSLT. Also, there is a full component API that allows developers to easily write their own components.
- Nux
- Nux is a toolkit making efficient and powerful XML processing easy. It is geared towards embedded use in high-throughput XML messaging middleware such as large-scale Peer-to-Peer infrastructures, message queues, publish-subscribe and matchmaking systems for Blogs/newsfeeds, text chat, data acquisition and distribution systems, application level routers, firewalls, classifiers, etc. Nux reliably processes whatever data fits into main memory (even, say, 250 MB messages), but it is not an XML database system, and does not attempt to be one. Nux integrates best-of-breed components, containing extensions of the XOM, Saxon and Lucene open-source libraries.
- KETL
- KETL is an extract, transform, and load (ETL) tool designed by Kinetic Networks. KETL includes job scheduling and alerting capabilities. The KETL Server is a Java-based data integration platform consisting of a multi-threaded server that manages various job executors. Jobs are defined using an XML definition language.
- Kettle
- K.E.T.T.L.E (Kettle ETTL Environment) is a meta-data driven ETTL tool. (ETTL: Extraction, Transformation, Transportation & Loading). No code has to be written to perform complex data transformations. Environment means that it is possible to create plugins to do custom transformations or access propriatary data sources. Kettle supports most databases on the market and has native support for slowly chaning dimensions on most platforms. The complete Kettle source code is over 160,000 lines of java code.
- Netflux
- Metadata based tool to allow for easier manipulations. Spring based configuration, BSF based scripting support, pluggable JDBC based data sources and sinks. A server and a GUI are planned.
- OpenDigger
- OpenDigger is a java based compiler for the xETL language. xETL is a language specifically projected to read, manipulate and write data in any format and database. With OpenDigger/XETL you can build Extraction-Transformation-Loading (ETL) programs virtually from and to any database platform.
- ServingXML
- ServingXML is a markup language for expressing XML pipelines, and an extensible Java framework for defining the elements of the language. It defines a vocabulary for expressing flat-XML, XML-flat, flat-flat, and XML-XML transformations in pipelines. ServingXML supports reading content as XML files, flat files, SQL queries or dynamically generated SAX events, transforming it with XSLT stylesheets and custom SAX filters, and writing it as XML, HTML, PDF or mail attachments. ServingXML is suited for converting flat file or database records to XML, with its support for namespaces, variant record types, multi-valued fields, segments and repeating groups, hierarchical grouping of records, and record-by-record validation with XML Schema.
- Talend
- Talend Open Studio is full-featured Data Integration OpenSource solution (ETL). Its graphical user interface, based on Eclipse Rich Client Platform (RCP) includes numerous components for business process modelling, as well as technical implementations of extracting, transformation and mapping of data flows. Data related script and underlying programs are generated in Perl and Java code.
- Scriptella
- Scriptella is an ETL and script execution tool. Its primary focus is simplicity. It doesn't require the user to learn another complex XML-based language to use it, but allows the use of SQL or another scripting language suitable for the data source to perform required transformations.
- ETL Integrator
- ETL (a highly unimaginative name) consists of 3 components. An ETL service engine that is a JBI compliant service engine implementation which can be deployed in a JBI container. An ETL Editor that is a design time netbeans module which allow users to design ETL process in a graphical way. An ETL Project that is a design time netbeans module which allows users to package ETL related artifacts in a jar file which could be deployed onto the ETL service engine.
- Jitterbit
- Jitterbit can act as a powerful ETL tool. Operations are defined, configured, and monitored with a GUI. The GUI can create document definitions, from simple flat file structures to complex hierarchic files structures. Jitterbit includes drag-and-drop mapping tool to transform data between your various system interfaces. Furthermore, one can set schedules, create success and failure events and track the results for your integration operations. Jitterbit supports Web Services, XML Files, HTTP/S, FTP, ODBC, Flat and Hierarchic file structures and file shares.
- Apatar
- Apatar integrates databases, files and applications. Apatar includes a visual job designer for defining mapping, joins, filtering, data validation and schedules. Connectors include MySQL, PostgreSQL, Oracle, MS SQL, Sybase, FTP, HTTP, SalesForce.com, SugarCRM, Compiere ERP, Goldmine CRM, XML, flat files, Webdav, Buzzsaw, LDAP, Amazon and Flickr. No coding is required to accomplish even a complex integration. All metadata is stored in XML.
- Spring Batch
- Spring Batch is a lightweight, comprehensive batch framework designed to enable the development of robust batch applications. Spring Batch provides reusable functions that are essential in processing large volumes of records, including logging/tracing, transaction management, job processing statistics, job restart, skip, and resource management. It also provides more advance technical services and features that will enable extremely high-volume and high performance batch jobs though optimization and partitioning techniques.
- JasperETL
- JasperETL was developed through a technology partnership with Talend. JasperETL includes Eclipse based user interfaces for process design, transformation mapping, debugging, process viewing. The project includes over 30 connectors like flat files, xml, databases, email, ftp and more. It includes wizards to help configure the processing of complex file formats including positional, delimited, CSV, RegExp, XML, and LDIF formatted data.
- Pentaho Data Integration
- Pentaho Data Integration provides a declarative approach to ETL where you specify what to do rather than how to do it. It includes a transformation library with over 70 mapping objects. In includes data warehousing capability for slowly changing and junk Dimensions. Includes support for multiple data sources including over 25 open source and proprietary database platforms, flat files, Excel documents, and more. The architecture is extensible with a plug-in mehcanism.
- Mural - Mural is an open source community with the purpose of developing an ecosystem of products that solve the problems in Master Data Management (MDM). Projects include: Master Index Studio which provides the supports the creation of a master index through the matching, de-duplication, merging, and cleansing . Data Integrator which provides extract, transform, load capability and a wide variety of data formats. Data Quality which features matching, standardization, profiling,and cleansing capabilities. Data Mashup Data Mashup which provides data mashup capability. Data Migrator which supports the migration of database objects across database instances
- Smooks
- Smooks provides a wide range of Data Transforms. Supports many different Source and Result types - XML/CSV/EDI/Java/JSON to XML/CSV/EDI/Java/JSON. It supports binding of Java Object Models from any data source. It is designed to process huge messages in the GByte range.
- Data Pipeline
- Data Pipeline provides data conversion, data processing, and data transformation. The toolkit has readers and writers for common file formats (CSV, Excel, Fixed-width, JDBC) along with decorators that can be chained together to process and transform data (filter, remove duplicates, lookups, validation).
Ubuntu9.04安装配置
$ sudo vi /etc/resolv.conf# Simple configuration file for xinetd## Some defaults, and include /etc/xinetd.d/defaults{# Please note that you need a log_type line to be able to use log_on_success# and log_on_failure. The default is the following :# log_type = SYSLOG daemon infoinstances = 60log_type = SYSLOG authprivlog_on_success = HOST PIDlog_on_failure = HOSTcps = 25 30}# default: on# description: The telnet server serves telnet sessions; it uses# unencrypted username/password pairs for authentication.service telnet{disable = noflags = REUSEsocket_type = streamwait = nouser = rootserver = /usr/sbin/in.telnetdlog_on_failure += USERID}LANG="zh_CN.GBK"
LANGUAGE="zh_CN:zh:en_US:en"
LC_CTYPE=zh_CN.GBK
LC_ALL=zh_CN.GBK
GST_ID3_TAG_ENCODING=GBK
ID3_TAG_ENCODING=GBK
在最后加入一句
export LANG=en_US
一、临时设置
export JAVA_HOME= /home/liupinghua/jdk1.5.0_18
打开~/.bashrc,添加行:
export JAVA_HOME= /home/liupinghua/jdk1.5.0_18
注销
这样每次以此用户登录Ubuntu,该环境变量都会生效。
$ vi /etc/profile
在里面加入:
export JAVA_HOME= /home/liupinghua/jdk1.5.0_18
注销
这样不管是以哪个用户登录,该环境变量都会生效。
tar -zxvf jsvc.tar.gz
cd jsvc-src
chmod +x configure
./configure --with-java=$JAVA_HOME
make
cd native
gedit Tomcat5.sh
--根据需要修改下面文件的参数
JAVA_HOME=/usr/lib/jvm/java-1.5.0-sun
CATALINA_HOME=/home/user/tomcat/
DAEMON_HOME=/home/user/tomcat/
TOMCAT_USER=user
#为tomcat设置一个启动用户,非root用户
#出于安全性考虑,如果是root用户,jsp执行权限太高,有被注入的问题
# for multi instances adapt those lines.
TMP_DIR=/var/tmp
PID_FILE=/var/run/jsvc.pid
CATALINA_BASE=/home/user/tomcat/
#CATALINA_OPTS="-Djava.library.path=/home/jfclere/jakarta-tomcat-connectors/jni/native/.libs"
CLASSPATH=
$JAVA_HOME/lib/tools.jar:
$CATALINA_HOME/bin/commons-daemon.jar:
$CATALINA_HOME/bin/bootstrap.jar
case "$1" in
start)
#
# Start Tomcat
#
$DAEMON_HOME/bin/jsvc-src/jsvc
-user $TOMCAT_USER
-home $JAVA_HOME
-Dcatalina.home=$CATALINA_HOME
-Dcatalina.base=$CATALINA_BASE
-Djava.io.tmpdir=$TMP_DIR
-wait 10
-outfile $CATALINA_HOME/logs/catalina.out
-errfile '&1'
$CATALINA_OPTS
-cp $CLASSPATH
org.apache.catalina.startup.Bootstrap
#
# To get a verbose JVM
#-verbose
# To get a debug of jsvc.
#-debug
exit $?
;;
stop)
#
# Stop Tomcat
#
$DAEMON_HOME/bin/jsvc-src/jsvc
-stop
org.apache.catalina.startup.Bootstrap
exit $?
;;
*)
echo "Usage tomcat.sh start/stop"
exit 1;;
esac
cp Tomcat5.sh /etc/init.d/tomcat.sh
修改执行权限
sudo chmod +x tomcat.sh
这样tomcat就会随着系统自动启用
测试:
sudo /etc/init.d/tomcat.sh start
sudo /etc/init.d/tomcat.sh stop
在slackware下有两个分区软件fdisk和cfdisk
例如我们已经有一个硬盘了,现在添加另一个硬盘到系统
那么我们根据命名规则知道这个新添加的硬盘应该是hdb。我们用下面命令给硬盘分区
你也可以用cfdisk来分区,命令如下
格式化硬盘
格式化成ext3格式
格式化成reiserfs的格式
让硬盘启动自动挂载
例如挂载/dev/hdb1分区到/mnt/hd目录下
用vi编辑/etc/fstab文件,加入如下内容
|
$ cat /etc/environment
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11: /usr/games" LANG="zh_CN.UTF-8" LANGUAGE="zh_CN:zh:en_US:en" |
因为有些情况下显示有问题, 所有root中强制设置成了LANG=C.
我收藏的链接(39)
我收藏的链接(38)
我收藏的链接(37)
我收藏的链接(36)
C# .net开发WebService客户端获取数据生成Word图表
调用WSDL生成WebService客户端代码:
C:\Program Files\Microsoft SDKs\Windows\v6.0A\Bin>wsdl http://172.150.131.20/pssys/services/EcFMISService?wsdl /n:namesp /out:filename.cs
Microsoft(R) Web Services 描述语言实用工具
[Microsoft (R) .NET Framework, Version 2.0.50727.1432]
Copyright (C) Microsoft Corporation. All rights reserved.
正在写入文件“filename.cs”。
添加引用:
COM中的Microsoft Graph 11.0 Object Library
COM中的Microsoft Office 11.0 Object Library
.NET中的System.Web.Services
.NET中的Microsoft.Office.Interop.Word
cs代码:
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using Word = Microsoft.Office.Interop.Word;
using System.Reflection;
namespace WindowsFormsApplication2
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
private void button1_Click(object sender, EventArgs e)
{
object oMissing = System.Reflection.Missing.Value;
object oEndOfDoc = "\\endofdoc"; /* \endofdoc is a predefined bookmark */
//Start Word and create a new document.
Word._Application oWord;
Word._Document oDoc;
oWord = new Word.Application();
oWord.Visible = true;
oDoc = oWord.Documents.Add(ref oMissing, ref oMissing,
ref oMissing, ref oMissing);
//Insert a paragraph at the beginning of the document.
Word.Paragraph oPara1;
oPara1 = oDoc.Content.Paragraphs.Add(ref oMissing);
oPara1.Range.Text = "Heading 1";
oPara1.Range.Font.Bold = 1;
oPara1.Format.SpaceAfter = 24; //24 pt spacing after paragraph.
oPara1.Range.InsertParagraphAfter();
//Insert a paragraph at the end of the document.
Word.Paragraph oPara2;
object oRng = oDoc.Bookmarks.get_Item(ref oEndOfDoc).Range;
oPara2 = oDoc.Content.Paragraphs.Add(ref oRng);
oPara2.Range.Text = "Heading 2";
oPara2.Format.SpaceAfter = 6;
oPara2.Range.InsertParagraphAfter();
//Insert another paragraph.
Word.Paragraph oPara3;
oRng = oDoc.Bookmarks.get_Item(ref oEndOfDoc).Range;
oPara3 = oDoc.Content.Paragraphs.Add(ref oRng);
oPara3.Range.Text = "This is a sentence of normal text. Now here is a table:";
oPara3.Range.Font.Bold = 0;
oPara3.Format.SpaceAfter = 24;
oPara3.Range.InsertParagraphAfter();
//Insert a 3 x 5 table, fill it with data, and make the first row
//bold and italic.
Word.Table oTable;
Word.Range wrdRng = oDoc.Bookmarks.get_Item(ref oEndOfDoc).Range;
oTable = oDoc.Tables.Add(wrdRng, 3, 5, ref oMissing, ref oMissing);
oTable.Range.ParagraphFormat.SpaceAfter = 6;
int r, c;
string strText;
for (r = 1; r <= 3; r++)
for (c = 1; c <= 5; c++)
{
strText = "r" + r + "c" + c;
oTable.Cell(r, c).Range.Text = strText;
}
oTable.Rows[1].Range.Font.Bold = 1;
oTable.Rows[1].Range.Font.Italic = 1;
//Add some text after the table.
Word.Paragraph oPara4;
oRng = oDoc.Bookmarks.get_Item(ref oEndOfDoc).Range;
oPara4 = oDoc.Content.Paragraphs.Add(ref oRng);
oPara4.Range.InsertParagraphBefore();
oPara4.Range.Text = "And here's another table:";
oPara4.Format.SpaceAfter = 24;
oPara4.Range.InsertParagraphAfter();
//Insert a 5 x 2 table, fill it with data, and change the column widths.
wrdRng = oDoc.Bookmarks.get_Item(ref oEndOfDoc).Range;
oTable = oDoc.Tables.Add(wrdRng, 5, 2, ref oMissing, ref oMissing);
oTable.Range.ParagraphFormat.SpaceAfter = 6;
for (r = 1; r <= 5; r++)
for (c = 1; c <= 2; c++)
{
strText = "r" + r + "c" + c;
oTable.Cell(r, c).Range.Text = strText;
}
oTable.Columns[1].Width = oWord.InchesToPoints(2); //Change width of columns 1 & 2
oTable.Columns[2].Width = oWord.InchesToPoints(3);
//Keep inserting text. When you get to 7 inches from top of the
//document, insert a hard page break.
object oPos;
double dPos = oWord.InchesToPoints(7);
oDoc.Bookmarks.get_Item(ref oEndOfDoc).Range.InsertParagraphAfter();
do
{
wrdRng = oDoc.Bookmarks.get_Item(ref oEndOfDoc).Range;
wrdRng.ParagraphFormat.SpaceAfter = 6;
wrdRng.InsertAfter("A line of text");
wrdRng.InsertParagraphAfter();
oPos = wrdRng.get_Information
(Word.WdInformation.wdVerticalPositionRelativeToPage);
}
while (dPos >= Convert.ToDouble(oPos));
object oCollapseEnd = Word.WdCollapseDirection.wdCollapseEnd;
object oPageBreak = Word.WdBreakType.wdPageBreak;
wrdRng.Collapse(ref oCollapseEnd);
wrdRng.InsertBreak(ref oPageBreak);
wrdRng.Collapse(ref oCollapseEnd);
wrdRng.InsertAfter("We're now on page 2. Here's my chart:");
wrdRng.InsertParagraphAfter();
//Insert a chart.
Word.InlineShape oShape;
object oClassType = "MSGraph.Chart.8";
wrdRng = oDoc.Bookmarks.get_Item(ref oEndOfDoc).Range;
oShape = wrdRng.InlineShapes.AddOLEObject(ref oClassType, ref oMissing,
ref oMissing, ref oMissing, ref oMissing,
ref oMissing, ref oMissing, ref oMissing);
//Demonstrate use of late bound oChart and oChartApp objects to
//manipulate the chart object with MSGraph.
object oChart;
object oChartApp;
oChart = oShape.OLEFormat.Object;
oChartApp = oChart.GetType().InvokeMember("Application",
BindingFlags.GetProperty, null, oChart, null);
//Change the chart type to Line.
object[] Parameters = new Object[1];
Parameters[0] = 4; //xlLine = 4
oChart.GetType().InvokeMember("ChartType", BindingFlags.SetProperty,
null, oChart, Parameters);
Graph.Chart objChart = (Graph.Chart)oShape.OLEFormat.Object;
// objChart.ChartType = Graph.XlChartType.xl3DPie;
//objChart.ChartType = Graph.XlChartType.xlColumnStacked;
objChart.ChartType = Graph.XlChartType.xlColumnClustered;
Graph.DataSheet dataSheet;
dataSheet = objChart.Application.DataSheet;
dataSheet.Cells[1, 2] = "第一季度";
dataSheet.Cells[1, 3] = "第二季度";
dataSheet.Cells[1, 4] = "第三季度";
dataSheet.Cells[1, 5] = "第四季度";
dataSheet.Cells[2, 1] = "东部";
dataSheet.Cells[2, 2] = "50";
dataSheet.Cells[2, 3] = "40";
dataSheet.Cells[2, 4] = "50";
dataSheet.Cells[2, 5] = "50";
dataSheet.Cells[3, 1] = "西部";
dataSheet.Cells[3, 2] = "60";
dataSheet.Cells[3, 3] = "70";
dataSheet.Cells[3, 4] = "80";
dataSheet.Cells[3, 5] = "60";
//dataSheet.Cells[3, 6] = "0";
dataSheet.Cells[4, 1] = "中部";
dataSheet.Cells[4, 2] = "50";
dataSheet.Cells[4, 3] = "40";
dataSheet.Cells[4, 4] = "50";
dataSheet.Cells[4, 5] = "50";
//dataSheet.Cells[4, 6] = "0";
objChart.Application.Update();
//Update the chart image and quit MSGraph.
oChartApp.GetType().InvokeMember("Update",
BindingFlags.InvokeMethod, null, oChartApp, null);
oChartApp.GetType().InvokeMember("Quit",
BindingFlags.InvokeMethod, null, oChartApp, null);
//... If desired, you can proceed from here using the Microsoft Graph
//Object model on the oChart and oChartApp objects to make additional
//changes to the chart.
//Set the width of the chart.
oShape.Width = oWord.InchesToPoints(6.25f);
oShape.Height = oWord.InchesToPoints(3.57f);
//Add text after the chart.
wrdRng = oDoc.Bookmarks.get_Item(ref oEndOfDoc).Range;
wrdRng.InsertParagraphAfter();
wrdRng.InsertAfter("THE END.");
//Close this form.
this.Close();
}
private void button2_Click(object sender, EventArgs e)
{
namesp.EcFMISService service = new namesp.EcFMISService();
System.Console.WriteLine(service.getDfShouldData("0104", "200806"));
}
}
}
T-SQL 编码标准最佳实践
Brian Walker
可能让人觉得很奇怪,但好像的确没有什么“正式的”T-SQL 编码标准。早在 1999 年末的时候,我惊喜地发现 John Hindmarsh 提出的 SQL Server 7.0 标准,我在 2000 年 2 月的社论中对他的某些建议进行了总结。(2000 年 2 月以及本月的“下载”中都包括了 John 原来的标准。)后来,Ron Talmage 撰写了一系列专栏文章,提出了他对各种“最佳方法”的建议,当然,SQL Server 小组也已正式发布了 SQL Server 最佳方法分析器 (SQLBPA)。现在,一位具有超过 25 年经验的数据库管理员和应用程序开发员 Brian Walker 又提出了他的建议和提示。
进 行 T-SQL 编程时常常会忽略编码标准,但这些标准却是开发小组顺利开展工作的关键工具。这里介绍的编码标准是我多年的开发成果。它们当然还没有得到普遍接受,而且不 可否认,有些标准带有主观色彩。我的目的实际上更多的是为了提高大家的意识,而不是吹捧自己是 T-SQL 样式方面的仲裁者:最重要的是要建立某些合理的编码标准并遵循这些标准。您在这篇文章中会发现有关 T-SQL 编程的一系列不同的编码标准、技巧和提示。它们并未以任何特定的优先级或重要性顺序列出。
让我们从格式开始。表面上,T-SQL 代码的格式似乎并不重要,但一致的格式可以使您的同事(不论是同一小组的成员还是更大范围的 T-SQL 开发团队的成员)更轻松地浏览和理解您的代码。T-SQL 语句有一个结构,遵循一目了然的结构使您可以更轻松地查找和确认语句的不同部分。统一的格式还使您可以更轻松地在复杂 T-SQL 语句中增删代码段,使调试工作变得更容易。下面是 SELECT 语句的格式示例:
SELECT C.Name
, E.NameLast
, E.NameFirst
, E.Number
, ISNULL(I.Description,'NA') AS Description
FROM tblCompany AS C
JOIN tblEmployee AS E
ON C.CompanyID = E.CompanyID
LEFT JOIN tblCoverage AS V
ON E.EmployeeID = V.EmployeeID
LEFT JOIN tblInsurance AS I
ON V.InsuranceID = I.InsuranceID
WHERE C.Name LIKE @Name
AND V.CreateDate > CONVERT(smalldatetime,
'01/01/2000')
ORDER BY C.Name
, E.NameLast
, E.NameFirst
, E.Number
, ISNULL(I.Description,'NA')
SELECT @Retain = @@ERROR, @Rows = @@ROWCOUNT
IF @Status = 0 SET @Status = @Retain
►一个嵌套代码块中的语句使用四个空格的缩进。(上述代码中的多行 SELECT 语句是一个 SQL 语句。)在同一语句中开始新行时,使 SQL 关键字右对齐。将代码编辑器配置为使用空格,而不是使用制表符。这样,不管使用何种程序查看代码,格式都是一致的。
►大写所有的 T-SQL 关键字,包括 T-SQL 函数。变量名称及光标名称使用混和大小写。数据类型使用小写。
►表名别名要简短,但意义要尽量明确。通常,使用大写的表名作为别名,使用 AS 关键字指定表或字段的别名。
►当一个 T-SQL 语句中涉及到多个表时,始终使用表名别名来限定字段名。这使其他人阅读起来更清楚,避免了含义模糊的引用。
►当相关数字出现在连续的代码行中时(例如一系列 SUBSTRING 函数调用),将它们排成列。这样容易浏览数字列表。
►使用一个(而不是两个)空行分隔 T-SQL 代码的逻辑块,只要需要就可以使用。
►声明 T-SQL 局部变量(例如 @lngTableID)时,使用适当的数据类型声明和一致的大写。
►始终指定字符数据类型的长度,并确保允许用户可能需要的最大字符数,因为超出最大长度的字符会丢失。
►始终指定十进制数据类型的精度和范围,否则,将默认为未指定精度和整数范围。
► 使用错误处理程序,但要记住行首 (BOL) 中的错误检查示例不会象介绍的那样起作用。用来检查 @@ERROR 系统函数的 T-SQL 语句 (IF) 实际上在进程中清除了 @@ERROR 值,无法再捕获除零之外的任何值。(即使示例起作用,它们也只能捕获最后发生的一个错误,而不是您更想捕获的第一个错误。)必须使用 SET 或 SELECT 立即捕获错误代码,如前面示例所示。如果状态变量仍然为零,应转换到状态变量。
►避免使用“未声明的”功能,例如系统表中未声明的列、T-SQL 语句中未声明的功能或者未声明的系统存储过程或扩展的存储过程。
►不要依 赖任何隐式的数据类型转换。例如,不能为数字变量赋予字符值,而假定 T-SQL 会进行必要的转换。相反,在为变量赋值或比较值之前,应使用适当的 CONVERT 函数使数据类型相匹配。另一个示例:虽然 T-SQL 会在进行比较之前对字符表达式进行隐式且自动的 RTRIM,但不能依赖此行为,因为兼容性级别设置非字符表达式会使情况复杂化。
►不要将空的变量值直接与比较运算符(符号)比较。如果变量可能为空,应使用 IS NULL 或 IS NOT NULL 进行比较,或者使用 ISNULL 函数。
►不要使用 STR 函数进行舍入,此函数只能用于整数。如果需要十进制值的字符串形式,应先使用 CONVERT 函数(转至不同的范围)或 ROUND 函数,然后将其转换为字符串。也可以使用 CEILING 和 FLOOR 函数。
►使用数学公式时要小心,因为 T-SQL 可能会将表达式强制理解为一个不需要的数据类型。如果需要十进制结果,应在整数常量后加点和零 (.0)。
►决不要依赖 SELECT 语句会按任何特定顺序返回行,除非在 ORDER BY 子句中指定了顺序。
► 通常,应将 ORDER BY 子句与 SELECT 语句一起使用。可预知的顺序(即使不是最方便的)比不可预知的顺序强,尤其是在开发或调试过程中。(部署到生产环境中之前,可能需要删除 ORDER BY 子句。)在返回行的顺序无关紧要的情况下,可以忽略 ORDER BY 的开销。
►不要在 T-SQL 代码中使用双引号。应为字符常量使用单引号。如果没有必要限定对象名称,可以使用(非 ANSI SQL 标准)括号将名称括起来。
►在 SQL Server 2000 中,尽量使用表变量来代替临时表。如果表变量包含大量数据,请注意索引非常有限(只有主键索引)。
►先在例程中创建临时表,最后再显式删除临时表。将 DDL 与 DML 语句混合使用有助于处理额外的重新编译活动。
►要认识到临时表并不是不可使用,适当地使用它们可以使某些例程更有效,例如,当需要重复引用大型表或常用表中的某个数据集时。但是,对于一次性事件,最好使用导出表。
►使用表值 UDF 时要小心,因为在变量(而不是常量)中传递某个参数时,如果在 WHERE 子句中使用该参数,会导致表扫描。还要避免在一个查询中多次使用相同的表值 UDF。但是,表值 UDF 确实具有某些非常方便的动态编译功能。[相关资料:参阅 Tom Moreau 在 2003 年 11 月份“生成序列号”专栏中的“使用 UDF 填充表变量”。-编者按]
►几乎所有的存储过程都应在开始时设置 SET NOCOUNT ON,而在结束时设置 SET NOCOUNT OFF。[SET NOCOUNT ON 使 SQL Server 无需在执行存储过程的每个语句后向客户端发送 DONE_IN_PROC 消息。- 编者按] 此标准同样适用于触发器。
►只要在例程中使用多个数据库修改语句,包括在一个循环中多次执行一个语句,就应考虑声明显式事务。
►使用基于光标的方法或临时表方法之前,应先寻找基于集的解决方案来解决问题。基于集的方法通常更有效。
► 与临时表一样,光标并不是不可使用。对小型数据集使用 FAST_FORWARD 光标通常要优于其他逐行处理方法,尤其是在必须引用几个表才能获得所需的数据时。在结果集中包括“合计”的例程通常要比使用光标执行的速度快。如果开发时 间允许,基于光标的方法和基于集的方法都可以尝试一下,看哪一种方法的效果更好。
►使用包含序号(从 1 到 N)的表很方便。
►理解 CROSS JOIN 的工作原理并加以利用。例如,您可以在工作数据表和序号表之间有效地使用 CROSS JOIN,结果集中将包含每个工作数据与序号组合的记录。
►我的结束语是:T-SQL 代码往往很简洁,因此如果某个代码块看起来很难处理或重复内容较多,那么可能存在一种更简单,更好的方法。
结论
如果您对我的建议有任何看法,欢迎随时向我发送电子邮件进行讨论,也可以就其他问题提出您的建议。我希望您将此作为谈话的开场白。
其他信息:摘自 Karen 2000 年 2 月份的社论
在 标准开发的前沿阵地上,有一股以 SQL Server 数据库管理员 John Hindmarsh 为首的独立的新生力量。MCT、MCSE 和 MCDBA 都是最值得您花时间去研究的。John 的贡献是撰写了一份详细的白皮书,概述了他对各种 SQL Server 相关标准提出的建议。我所知道的其他唯一提出类似建议的文章是 Andrew Zanevsky 的《Transact-SQL Programming》(ISBN 1-56592-401-0) 中的“Format and Style”一章。Andrew、SQL Server Professional 的投稿人 Tom Moreau 和 Paul Munkenbeck 以及 John 的朋友兼同事 Stephen James 都为 John 的白皮书做出过贡献。下面是 John 为编写存储过程提供的建议示例:
| • |
使用 SQL-92 标准连接句法。 |
| • |
为了提高性能,应优先使用连接,然后使用子查询或嵌套查询。 |
| • |
确保变量和参数的类型和大小与表数据列相匹配。 |
| • |
确保使用所有变量和参数,或者全部删除。 |
| • |
尽可能将临时对象放置在本地。 |
| • |
只使用在存储过程中创建的临时表。 |
| • |
检查输入参数的有效性。 |
| • |
优先使用 SELECT...INTO,然后使用 INSERT...SELECT,以避免大量死锁。 |
| • |
维护工作需要的逻辑单元;在可以缩短的情况下,不要创建大量或长时间运行的进程。 |
| • |
不要在任何代码中使用 SELECT *。 |
| • |
在过程中使用缩进、块、制表符和空格(参阅示例脚本)。 |
| • |
T-SQL 语句要大写。 |
| • |
在过程中添加大量注释,确保可以识别进程。在有助于澄清处理步骤的地方使用行注释。 |
| • |
包括事务管理,除非要从 MTS 进程中调用过程。(为 MTS 进程编写独立的过程。) |
| • |
监视 @@TRANCOUNT 以确定事务的责任级别。 |
| • |
避免使用 GOTO,错误处理程序中除外。 |
| • |
避免使用嵌套过程。 |
| • |
避免隐式解析对象名称,确保所有对象都归 dbo 所有。 |
Windows Mobile可用的JVM
Esmertec Jbed
phoneme,支持JSR82,JSR179等很多规范
PhoneMe有一个已经打好的cab安装包:
http://www.cs.kuleuven.be/~davy/phoneme/?q=node/10
J2ME / JavaME Midlet Manager in Pocket PC PDA
Mysaifu JVM
JMM
IBM J9
如何扩展Ubuntu的/root分区
除了使用Ubuntu livecd还可以直接使用GParted LiveCD
如果是使用lvm,可以,将pv将入vg,然后调整root filesystem使用的lv的大小
Ubuntu下Flash中文方格乱码的解决
- 删除/etc/fonts/conf.d/49-sansserif.conf
sudo mv /etc/fonts/conf.d/49-sansserif.conf /etc/fonts/conf.d/49-sansserif.conf_back
- 修改/etc/fonts/conf.d/49-sansserif.conf
输入:
cd /etc/fonts/conf.d/为了安全,备份一下:
sudo cp 49-sansserif.conf 49-sansserif.conf_backup
输入如下指令:
sudo gedit ./49-sansserif.conf
此时文件显示内容。
将其中的第1、2、4个后面的sans-serif或者serif用你自己系统中支持中文的字体的名字代替,注意字体名字的大小写
比如:我的系统中安装了wqy-zenhei.ttf,我则用wqy-zenhei代替上述所说的字段,结果如下:
<match target=”pattern”>
<test qual=”all” name=”family” compare=”not_eq”>
<string>wqy-zenhei</string>
</test>
<test qual=”all” name=”family” compare=”not_eq”>
<string>wqy-zenhei</string>
</test>
<test qual=”all” name=”family” compare=”not_eq”>
<string>monospace</string>
</test>
<edit name=”family” mode=”append_last”>
<string>wqy-zenhei</string>
</edit>
</match>

