Changeset 2549

Timestamp:

10/28/08 16:19:22 (2 months ago)

Author:

huangb

Message:

This version works fine. Have to put an empty line before the first stop word as an alternative solution, since there's a "?" before the stop word "a".

Files:

• trunk/plugins/preprocessing/edu.iu.nwb.preprocessing.text.normalization/src/edu/iu/nwb/preprocessing/text/normalization/StandardNormalyzer.java (modified) (2 diffs)
• trunk/plugins/preprocessing/edu.iu.nwb.preprocessing.text.normalization/src/edu/iu/nwb/preprocessing/text/normalization/stopwords.txt (modified) (1 diff)

trunk/plugins/preprocessing/edu.iu.nwb.preprocessing.text.normalization/src/edu/iu/nwb/preprocessing/text/normalization/StandardNormalyzer.java

@@ -77 +77 @@
-
-    private String[] getStopWords() {
-
+-2
-        InputStream is = null;
-        BufferedReader br = null;
@@ -87 +87 @@
-                final ClassLoader loader = getClass().getClassLoader();
-                is = loader.getResourceAsStream (filePath);
-
+, "UTF-8"
-            while (null != (line = br.readLine())) {
-                 list.add(line);

trunk/plugins/preprocessing/edu.iu.nwb.preprocessing.text.normalization/src/edu/iu/nwb/preprocessing/text/normalization/stopwords.txt

@@ -1 @@
-
+
+
-about
-above