--- tags: - sentence-transformers - sentence-similarity - feature-extraction - dense - generated_from_trainer - dataset_size:10000 - loss:CosineSimilarityLoss base_model: microsoft/codebert-base widget: - source_sentence: "private void createTree(DefaultMutableTreeNode top) throws MalformedURLException,\ \ ParserConfigurationException, SAXException, IOException {\n InputStream\ \ stream;\n URL url = new URL(SHIPS_URL + view.getBaseurl());\n \ \ try {\n stream = url.openStream();\n } catch (Exception e)\ \ {\n stream = getClass().getResourceAsStream(\"ships.xml\");\n \ \ }\n DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();\n\ \ DocumentBuilder parser = factory.newDocumentBuilder();\n Document\ \ doc = parser.parse(stream);\n NodeList races = doc.getElementsByTagName(\"\ race\");\n for (int i = 0; i < races.getLength(); i++) {\n Element\ \ race = (Element) races.item(i);\n top.add(buildRaceTree(race));\n\ \ }\n top.setUserObject(\"Ships\");\n view.getShipTree().repaint();\n\ \ view.getShipTree().expandRow(0);\n }" sentences: - "private GenomicSequence fetch(Chromosome k, int start, int end) throws IOException\ \ {\n try {\n String chr = k.toString();\n if (chr.toLowerCase().startsWith(\"\ chr\")) chr = chr.substring(3);\n SAXParserFactory f = SAXParserFactory.newInstance();\n\ \ f.setNamespaceAware(false);\n f.setValidating(false);\n\ \ SAXParser parser = f.newSAXParser();\n URL url = new URL(\"\ http://genome.ucsc.edu/cgi-bin/das/\" + genomeVersion + \"/dna?segment=\" + URLEncoder.encode(chr,\ \ \"UTF-8\") + \":\" + (start + 1) + \",\" + (end));\n DASHandler handler\ \ = new DASHandler();\n InputStream in = url.openStream();\n \ \ parser.parse(in, handler);\n in.close();\n GenomicSequence\ \ seq = new GenomicSequence();\n seq.sequence = handler.bytes.toByteArray();\n\ \ seq.start = start;\n seq.end = end;\n if (seq.sequence.length\ \ != seq.length()) throw new IOException(\"bad bound \" + seq + \" \" + seq.sequence.length\ \ + \" \" + seq.length());\n return seq;\n } catch (IOException\ \ err) {\n throw err;\n } catch (Exception e) {\n \ \ throw new IOException(e);\n }\n }" - "public static void main(String[] args) throws Exception {\n FileChannel\ \ fc = new FileOutputStream(\"data.txt\").getChannel();\n fc.write(ByteBuffer.wrap(\"\ some text \".getBytes()));\n fc.close();\n fc = new RandomAccessFile(\"\ data.txt\", \"rw\").getChannel();\n fc.position(fc.size());\n fc.write(ByteBuffer.wrap(\"\ some more\".getBytes()));\n fc.close();\n fc = new FileInputStream(\"\ data.txt\").getChannel();\n ByteBuffer buff = ByteBuffer.allocate(BSIZE);\n\ \ fc.read(buff);\n buff.flip();\n while (buff.hasRemaining())\ \ {\n PrintUtil.prt((char) buff.get());\n }\n }" - "private FileLog(LOG_LEVEL displayLogLevel, LOG_LEVEL logLevel, String logPath)\ \ {\n this.logLevel = logLevel;\n this.displayLogLevel = displayLogLevel;\n\ \ if (null != logPath) {\n logFile = new File(logPath, \"current.log\"\ );\n log(LOG_LEVEL.DEBUG, \"FileLog\", \"Initialising logfile \" +\ \ logFile.getAbsolutePath() + \" .\");\n try {\n if\ \ (logFile.exists()) {\n if (!logFile.renameTo(new File(logPath,\ \ System.currentTimeMillis() + \".log\"))) {\n File newFile\ \ = new File(logPath, System.currentTimeMillis() + \".log\");\n \ \ if (newFile.exists()) {\n log(LOG_LEVEL.WARN,\ \ \"FileLog\", \"The file (\" + newFile.getAbsolutePath() + newFile.getName()\ \ + \") already exists, will overwrite it.\");\n newFile.delete();\n\ \ }\n newFile.createNewFile();\n\ \ FileInputStream inStream = new FileInputStream(logFile);\n\ \ FileOutputStream outStream = new FileOutputStream(newFile);\n\ \ byte buffer[] = null;\n int offSet\ \ = 0;\n while (inStream.read(buffer, offSet, 2048) !=\ \ -1) {\n outStream.write(buffer);\n \ \ offSet += 2048;\n }\n \ \ inStream.close();\n outStream.close();\n \ \ logFile.delete();\n logFile = new File(logPath,\ \ \"current.log\");\n }\n }\n \ \ logFile.createNewFile();\n } catch (IOException e) {\n \ \ logFile = null;\n }\n } else {\n logFile\ \ = null;\n }\n }" - source_sentence: "private ByteBuffer readProgram(URL url) throws IOException {\n\ \ StringBuilder program = new StringBuilder();\n BufferedReader\ \ in = null;\n try {\n in = new BufferedReader(new InputStreamReader(url.openStream()));\n\ \ String line;\n while ((line = in.readLine()) != null)\ \ {\n program.append(line).append(\"\\n\");\n }\n \ \ } finally {\n if (in != null) in.close();\n }\n \ \ ByteBuffer buffer = BufferUtils.createByteBuffer(program.length());\n \ \ for (int i = 0; i < program.length(); i++) {\n buffer.put((byte)\ \ (program.charAt(i) & 0xFF));\n }\n buffer.flip();\n return\ \ buffer;\n }" sentences: - "public int delete(BusinessObject o) throws DAOException {\n int delete\ \ = 0;\n Currency curr = (Currency) o;\n try {\n PreparedStatement\ \ pst = connection.prepareStatement(XMLGetQuery.getQuery(\"DELETE_CURRENCY\"));\n\ \ pst.setInt(1, curr.getId());\n delete = pst.executeUpdate();\n\ \ if (delete <= 0) {\n connection.rollback();\n \ \ throw new DAOException(\"Number of rows <= 0\");\n } else\ \ if (delete > 1) {\n connection.rollback();\n throw\ \ new DAOException(\"Number of rows > 1\");\n }\n connection.commit();\n\ \ } catch (SQLException e) {\n Log.write(e.getMessage());\n\ \ throw new DAOException(\"A SQLException has occured\");\n \ \ } catch (NullPointerException npe) {\n Log.write(npe.getMessage());\n\ \ throw new DAOException(\"Connection null\");\n }\n \ \ return delete;\n }" - "public static String encrypt(String plaintext) throws EncryptionException {\n\ \ if (plaintext == null) {\n throw new EncryptionException();\n\ \ }\n try {\n final MessageDigest md = MessageDigest.getInstance(\"\ SHA\");\n md.update(plaintext.getBytes(\"UTF-8\"));\n return\ \ Base64.encodeBytes(md.digest());\n } catch (NoSuchAlgorithmException\ \ e) {\n throw new EncryptionException(e);\n } catch (UnsupportedEncodingException\ \ e) {\n throw new EncryptionException(e);\n }\n }" - "public static void copyFile(File in, File out) throws IOException {\n \ \ FileChannel sourceChannel = new FileInputStream(in).getChannel();\n FileChannel\ \ destinationChannel = new FileOutputStream(out).getChannel();\n destinationChannel.transferFrom(sourceChannel,\ \ 0, sourceChannel.size());\n sourceChannel.close();\n destinationChannel.close();\n\ \ }" - source_sentence: "private void sortWhats(String[] labels, int[] whats, String simplifyString)\ \ {\n int n = whats.length;\n boolean swapped;\n do {\n \ \ swapped = false;\n for (int i = 0; i < n - 1; i++) {\n\ \ int i0_pos = simplifyString.indexOf(labels[whats[i]]);\n \ \ int i1_pos = simplifyString.indexOf(labels[whats[i + 1]]);\n \ \ if (i0_pos > i1_pos) {\n int temp = whats[i];\n\ \ whats[i] = whats[i + 1];\n whats[i + 1]\ \ = temp;\n swapped = true;\n }\n \ \ }\n } while (swapped);\n }" sentences: - "public String GetMemberName(String id) {\n String name = null;\n \ \ try {\n String line;\n URL url = new URL(intvasmemberDeatails\ \ + \"?CID=\" + id);\n URLConnection connection = url.openConnection();\n\ \ BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));\n\ \ while ((line = reader.readLine()) != null) {\n name\ \ = line;\n }\n } catch (MalformedURLException e) {\n \ \ e.printStackTrace();\n } catch (IOException e) {\n e.printStackTrace();\n\ \ }\n String[] parts = name.split(\" \");\n rating = parts[2];\n\ \ return parts[0] + \" \" + parts[1];\n }" - "private static void readAndRewrite(File inFile, File outFile) throws IOException\ \ {\n ImageInputStream iis = ImageIO.createImageInputStream(new BufferedInputStream(new\ \ FileInputStream(inFile)));\n DcmParser dcmParser = DcmParserFactory.getInstance().newDcmParser(iis);\n\ \ Dataset ds = DcmObjectFactory.getInstance().newDataset();\n dcmParser.setDcmHandler(ds.getDcmHandler());\n\ \ dcmParser.parseDcmFile(null, Tags.PixelData);\n PixelDataReader\ \ pdReader = pdFact.newReader(ds, iis, dcmParser.getDcmDecodeParam().byteOrder,\ \ dcmParser.getReadVR());\n System.out.println(\"reading \" + inFile +\ \ \"...\");\n pdReader.readPixelData(false);\n ImageOutputStream\ \ out = ImageIO.createImageOutputStream(new BufferedOutputStream(new FileOutputStream(outFile)));\n\ \ DcmEncodeParam dcmEncParam = DcmEncodeParam.IVR_LE;\n ds.writeDataset(out,\ \ dcmEncParam);\n ds.writeHeader(out, dcmEncParam, Tags.PixelData, dcmParser.getReadVR(),\ \ dcmParser.getReadLength());\n System.out.println(\"writing \" + outFile\ \ + \"...\");\n PixelDataWriter pdWriter = pdFact.newWriter(pdReader.getPixelDataArray(),\ \ false, ds, out, dcmParser.getDcmDecodeParam().byteOrder, dcmParser.getReadVR());\n\ \ pdWriter.writePixelData();\n out.flush();\n out.close();\n\ \ System.out.println(\"done!\");\n }" - "public void insertComponent() throws SQLException {\n Connection connection\ \ = null;\n PreparedStatement ps = null;\n try {\n Class.forName(\"\ com.mysql.jdbc.Driver\");\n connection = (Connection) DriverManager.getConnection(this.jdbcURL);\n\ \ connection.setAutoCommit(false);\n String query = \"INSERT\ \ INTO components(name,rate,quantity, description) VALUES(?,?,?,?)\";\n \ \ ps = (PreparedStatement) connection.prepareStatement(query);\n \ \ ps.setString(1, this.name);\n ps.setDouble(2, this.rate);\n \ \ ps.setInt(3, this.quantity);\n ps.setString(4, this.description);\n\ \ ps.executeUpdate();\n connection.commit();\n }\ \ catch (Exception ex) {\n connection.rollback();\n } finally\ \ {\n try {\n connection.close();\n } catch\ \ (Exception ex) {\n }\n try {\n ps.close();\n\ \ } catch (Exception ex) {\n }\n }\n }" - source_sentence: "public static String toMD5(String seed) {\n MessageDigest\ \ md5 = null;\n StringBuffer temp_sb = null;\n try {\n \ \ md5 = MessageDigest.getInstance(\"MD5\");\n md5.update(seed.getBytes());\n\ \ byte[] array = md5.digest();\n temp_sb = new StringBuffer();\n\ \ for (int i = 0; i < array.length; i++) {\n int b =\ \ array[i] & 0xFF;\n if (b < 0x10) temp_sb.append('0');\n \ \ temp_sb.append(Integer.toHexString(b));\n }\n }\ \ catch (NoSuchAlgorithmException err) {\n err.printStackTrace();\n\ \ }\n return temp_sb.toString();\n }" sentences: - "public static boolean checkEncryptedPassword(String md5key, String passwordAccount,\ \ String encryptedPassword, int passwdenc) throws NoSuchAlgorithmException, UnsupportedEncodingException\ \ {\n MessageDigest md = MessageDigest.getInstance(Constants.ALGORITHM);\n\ \ switch(passwdenc) {\n case 1:\n md.update((md5key\ \ + encryptedPassword).getBytes(\"8859_1\"));\n return md.digest().equals(passwordAccount.getBytes(\"\ 8859_1\"));\n case 2:\n md.update((encryptedPassword\ \ + md5key).getBytes(\"8859_1\"));\n return md.digest().equals(passwordAccount.getBytes(\"\ 8859_1\"));\n default:\n return false;\n }\n\ \ }" - "@Test\n public void shouldDownloadFileUsingPublicLink() throws Exception {\n\ \ String bucketName = \"test-\" + UUID.randomUUID();\n Service service\ \ = new WebClientService(credentials);\n service.createBucket(bucketName);\n\ \ File file = folder.newFile(\"foo.txt\");\n FileUtils.writeStringToFile(file,\ \ UUID.randomUUID().toString());\n service.createObject(bucketName, file.getName(),\ \ file, new NullProgressListener());\n String publicUrl = service.getPublicUrl(bucketName,\ \ file.getName(), new DateTime().plusDays(5));\n File saved = folder.newFile(\"\ saved.txt\");\n InputStream input = new URL(publicUrl).openConnection().getInputStream();\n\ \ FileOutputStream output = new FileOutputStream(saved);\n IOUtils.copy(input,\ \ output);\n output.close();\n assertThat(\"Corrupted download\"\ , Files.computeMD5(saved), equalTo(Files.computeMD5(file)));\n service.deleteObject(bucketName,\ \ file.getName());\n service.deleteBucket(bucketName);\n }" - "public static void main(String[] args) {\n try {\n boolean\ \ readExp = Utils.getFlag('l', args);\n final boolean writeExp = Utils.getFlag('s',\ \ args);\n final String expFile = Utils.getOption('f', args);\n \ \ if ((readExp || writeExp) && (expFile.length() == 0)) {\n \ \ throw new Exception(\"A filename must be given with the -f option\");\n\ \ }\n Experiment exp = null;\n if (readExp) {\n\ \ FileInputStream fi = new FileInputStream(expFile);\n \ \ ObjectInputStream oi = new ObjectInputStream(new BufferedInputStream(fi));\n\ \ exp = (Experiment) oi.readObject();\n oi.close();\n\ \ } else {\n exp = new Experiment();\n }\n\ \ System.err.println(\"Initial Experiment:\\n\" + exp.toString());\n\ \ final JFrame jf = new JFrame(\"Weka Experiment Setup\");\n \ \ jf.getContentPane().setLayout(new BorderLayout());\n final SetupPanel\ \ sp = new SetupPanel();\n jf.getContentPane().add(sp, BorderLayout.CENTER);\n\ \ jf.addWindowListener(new WindowAdapter() {\n\n public\ \ void windowClosing(WindowEvent e) {\n System.err.println(\"\ \\nFinal Experiment:\\n\" + sp.m_Exp.toString());\n if (writeExp)\ \ {\n try {\n FileOutputStream\ \ fo = new FileOutputStream(expFile);\n ObjectOutputStream\ \ oo = new ObjectOutputStream(new BufferedOutputStream(fo));\n \ \ oo.writeObject(sp.m_Exp);\n oo.close();\n\ \ } catch (Exception ex) {\n \ \ ex.printStackTrace();\n System.err.println(\"Couldn't\ \ write experiment to: \" + expFile + '\\n' + ex.getMessage());\n \ \ }\n }\n jf.dispose();\n \ \ System.exit(0);\n }\n });\n \ \ jf.pack();\n jf.setVisible(true);\n System.err.println(\"\ Short nap\");\n Thread.currentThread().sleep(3000);\n System.err.println(\"\ Done\");\n sp.setExperiment(exp);\n } catch (Exception ex) {\n\ \ ex.printStackTrace();\n System.err.println(ex.getMessage());\n\ \ }\n }" - source_sentence: "protected void serveStaticContent(HttpServletRequest request,\ \ HttpServletResponse response, String pathInfo) throws ServletException {\n \ \ InputStream is = servletConfig.getServletContext().getResourceAsStream(pathInfo);\n\ \ if (is == null) {\n throw new ServletException(\"Static resource\ \ \" + pathInfo + \" is not available\");\n }\n try {\n \ \ int ind = pathInfo.lastIndexOf(\".\");\n if (ind != -1 && ind\ \ < pathInfo.length()) {\n String type = STATIC_CONTENT_TYPES.get(pathInfo.substring(ind\ \ + 1));\n if (type != null) {\n response.setContentType(type);\n\ \ }\n }\n ServletOutputStream os = response.getOutputStream();\n\ \ IOUtils.copy(is, os);\n os.flush();\n } catch (IOException\ \ ex) {\n throw new ServletException(\"Static resource \" + pathInfo\ \ + \" can not be written to the output stream\");\n }\n }" sentences: - "private static String makeLoginRequest(String authenticationUrl, String accountEmail,\ \ String accountPassword) throws IOException {\n URL url = new URL(authenticationUrl);\n\ \ HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();\n\ \ urlConnection.setRequestMethod(\"POST\");\n urlConnection.setDoInput(true);\n\ \ urlConnection.setDoOutput(true);\n urlConnection.setUseCaches(false);\n\ \ urlConnection.setRequestProperty(\"Content-Type\", \"application/x-www-form-urlencoded\"\ );\n StringBuffer content = new StringBuffer();\n content.append(\"\ Email=\").append(URLEncoder.encode(accountEmail, \"UTF-8\"));\n content.append(\"\ &Passwd=\").append(URLEncoder.encode(accountPassword, \"UTF-8\"));\n content.append(\"\ &source=\").append(URLEncoder.encode(\"Google Base data API for OFBiz\", \"UTF-8\"\ ));\n content.append(\"&service=\").append(URLEncoder.encode(\"gbase\"\ , \"UTF-8\"));\n OutputStream outputStream = urlConnection.getOutputStream();\n\ \ outputStream.write(content.toString().getBytes(\"UTF-8\"));\n \ \ outputStream.close();\n int responseCode = urlConnection.getResponseCode();\n\ \ InputStream inputStream;\n if (responseCode == HttpURLConnection.HTTP_OK)\ \ {\n inputStream = urlConnection.getInputStream();\n } else\ \ {\n inputStream = urlConnection.getErrorStream();\n }\n \ \ return toString(inputStream);\n }" - "public void constructFundamentalView() {\n String className;\n \ \ String methodName;\n String field;\n boolean foundRead = false;\n\ \ boolean foundWrite = false;\n boolean classWritten = false;\n\ \ try {\n FundView = new BufferedWriter(new FileWriter(\"InfoFiles/FundamentalView.txt\"\ ));\n FileInputStream fstreamPC = new FileInputStream(\"InfoFiles/PrincipleClassGroup.txt\"\ );\n DataInputStream inPC = new DataInputStream(fstreamPC);\n \ \ BufferedReader PC = new BufferedReader(new InputStreamReader(inPC));\n\ \ while ((field = PC.readLine()) != null) {\n className\ \ = field;\n FundView.write(className);\n FundView.newLine();\n\ \ classWritten = true;\n while ((methodName = PC.readLine())\ \ != null) {\n if (methodName.contentEquals(\"EndOfClass\"\ )) break;\n FundView.write(\"StartOfMethod\");\n \ \ FundView.newLine();\n FundView.write(methodName);\n\ \ FundView.newLine();\n for (int i = 0;\ \ i < readFileCount && foundRead == false; i++) {\n if\ \ (methodName.compareTo(readArray[i]) == 0) {\n foundRead\ \ = true;\n for (int j = 1; readArray[i + j].compareTo(\"\ EndOfMethod\") != 0; j++) {\n if (readArray[i +\ \ j].indexOf(\".\") > 0) {\n field = readArray[i\ \ + j].substring(0, readArray[i + j].indexOf(\".\"));\n \ \ if (field.compareTo(className) == 0) {\n \ \ FundView.write(readArray[i + j]);\n \ \ FundView.newLine();\n }\n\ \ }\n }\n \ \ }\n }\n for (int i = 0; i\ \ < writeFileCount && foundWrite == false; i++) {\n if\ \ (methodName.compareTo(writeArray[i]) == 0) {\n foundWrite\ \ = true;\n for (int j = 1; writeArray[i + j].compareTo(\"\ EndOfMethod\") != 0; j++) {\n if (writeArray[i\ \ + j].indexOf(\".\") > 0) {\n field = writeArray[i\ \ + j].substring(0, writeArray[i + j].indexOf(\".\"));\n \ \ if (field.compareTo(className) == 0) {\n \ \ FundView.write(writeArray[i + j]);\n \ \ FundView.newLine();\n }\n\ \ }\n }\n \ \ }\n }\n FundView.write(\"\ EndOfMethod\");\n FundView.newLine();\n \ \ foundRead = false;\n foundWrite = false;\n \ \ }\n if (classWritten == true) {\n FundView.write(\"\ EndOfClass\");\n FundView.newLine();\n classWritten\ \ = false;\n }\n }\n PC.close();\n \ \ FundView.close();\n } catch (IOException e) {\n e.printStackTrace();\n\ \ }\n }" - "public static void main(String[] args) throws Exception {\n if (args.length\ \ != 2) {\n System.out.println(\"arguments: sourcefile destfile\");\n\ \ System.exit(1);\n }\n FileChannel in = new FileInputStream(args[0]).getChannel(),\ \ out = new FileOutputStream(args[1]).getChannel();\n in.transferTo(0,\ \ in.size(), out);\n }" pipeline_tag: sentence-similarity library_name: sentence-transformers metrics: - pearson_cosine - spearman_cosine model-index: - name: SentenceTransformer based on microsoft/codebert-base results: - task: type: semantic-similarity name: Semantic Similarity dataset: name: bigclonebench validation type: bigclonebench-validation metrics: - type: pearson_cosine value: 0.9050117464281175 name: Pearson Cosine - type: spearman_cosine value: 0.8342891483192489 name: Spearman Cosine --- # SentenceTransformer based on microsoft/codebert-base This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [microsoft/codebert-base](https://huggingface.co/microsoft/codebert-base). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [microsoft/codebert-base](https://huggingface.co/microsoft/codebert-base) - **Maximum Sequence Length:** 256 tokens - **Output Dimensionality:** 768 dimensions - **Similarity Function:** Cosine Similarity - **Supported Modality:** Text ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'RobertaModel'}) (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True}) ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("wraithklu/codesentinel-clone-detector") # Run inference sentences = [ 'protected void serveStaticContent(HttpServletRequest request, HttpServletResponse response, String pathInfo) throws ServletException {\n InputStream is = servletConfig.getServletContext().getResourceAsStream(pathInfo);\n if (is == null) {\n throw new ServletException("Static resource " + pathInfo + " is not available");\n }\n try {\n int ind = pathInfo.lastIndexOf(".");\n if (ind != -1 && ind < pathInfo.length()) {\n String type = STATIC_CONTENT_TYPES.get(pathInfo.substring(ind + 1));\n if (type != null) {\n response.setContentType(type);\n }\n }\n ServletOutputStream os = response.getOutputStream();\n IOUtils.copy(is, os);\n os.flush();\n } catch (IOException ex) {\n throw new ServletException("Static resource " + pathInfo + " can not be written to the output stream");\n }\n }', 'private static String makeLoginRequest(String authenticationUrl, String accountEmail, String accountPassword) throws IOException {\n URL url = new URL(authenticationUrl);\n HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();\n urlConnection.setRequestMethod("POST");\n urlConnection.setDoInput(true);\n urlConnection.setDoOutput(true);\n urlConnection.setUseCaches(false);\n urlConnection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");\n StringBuffer content = new StringBuffer();\n content.append("Email=").append(URLEncoder.encode(accountEmail, "UTF-8"));\n content.append("&Passwd=").append(URLEncoder.encode(accountPassword, "UTF-8"));\n content.append("&source=").append(URLEncoder.encode("Google Base data API for OFBiz", "UTF-8"));\n content.append("&service=").append(URLEncoder.encode("gbase", "UTF-8"));\n OutputStream outputStream = urlConnection.getOutputStream();\n outputStream.write(content.toString().getBytes("UTF-8"));\n outputStream.close();\n int responseCode = urlConnection.getResponseCode();\n InputStream inputStream;\n if (responseCode == HttpURLConnection.HTTP_OK) {\n inputStream = urlConnection.getInputStream();\n } else {\n inputStream = urlConnection.getErrorStream();\n }\n return toString(inputStream);\n }', 'public static void main(String[] args) throws Exception {\n if (args.length != 2) {\n System.out.println("arguments: sourcefile destfile");\n System.exit(1);\n }\n FileChannel in = new FileInputStream(args[0]).getChannel(), out = new FileOutputStream(args[1]).getChannel();\n in.transferTo(0, in.size(), out);\n }', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 768] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities) # tensor([[ 1.0000, -0.0114, 0.9605], # [-0.0114, 1.0000, 0.0068], # [ 0.9605, 0.0068, 1.0000]]) ``` ## Evaluation ### Metrics #### Semantic Similarity * Dataset: `bigclonebench-validation` * Evaluated with [EmbeddingSimilarityEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.sentence_transformer.evaluation.EmbeddingSimilarityEvaluator) | Metric | Value | |:--------------------|:-----------| | pearson_cosine | 0.905 | | **spearman_cosine** | **0.8343** | ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 10,000 training samples * Columns: sentence_0, sentence_1, and label * Approximate statistics based on the first 100 samples: | | sentence_0 | sentence_1 | label | |:---------|:--------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | modality | text | text | | | details | | | | * Samples: | sentence_0 | sentence_1 | label | |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------| | public InputLattice(String file, int type, boolean convertJoinMeet) throws FileNotFoundException, IOException {
InputStream latStream = null;
labels = new Vector();
upperCoversVector = new Vector();
Vector upperCovers = new Vector();
edgeColors = new Hashtable();
Vector upperCovering = new Vector();
int level = 0;
String str;
String currentElt = null;
String cover = null;
boolean first = true;
StreamTokenizer in;
switch(type) {
case FILE:
latStream = new FileInputStream(file);
break;
case STRING:
latStream = new StringBufferInputStream(file);
break;
case URL:
URL url = new URL("http", Config.HOST, Config.PORT, "/" + Config.HASSE + "/" + Config.URL_STREAMER + file);
System.out.println("url is " + url);
URLConnection connection = url.openCo...
| private String calculateHash(String s) {
if (s == null) {
return null;
}
MessageDigest messageDigest;
try {
messageDigest = MessageDigest.getInstance("SHA");
} catch (NoSuchAlgorithmException e) {
logger.error("Could not find a message digest algorithm.");
return null;
}
messageDigest.update(s.getBytes());
byte[] hash = messageDigest.digest();
StringBuilder sb = new StringBuilder();
for (byte b : hash) {
sb.append(String.format("%02x", b));
}
return sb.toString();
}
| 0.0 | | void load(URL url) throws IOException {
BufferedReader r = new BufferedReader(new InputStreamReader(url.openStream()));
Vector3f scale = new Vector3f(1, 1, 1);
Group currentGroup = new Group();
currentGroup.name = "default";
groups.add(currentGroup);
String line;
while ((line = r.readLine()) != null) {
String[] params = line.split(" +");
if (params.length == 0) continue;
String command = params[0];
if (params[0].equals("v")) {
Vector3f vertex = new Vector3f(Float.parseFloat(params[1]) * scale.x, Float.parseFloat(params[2]) * scale.y, Float.parseFloat(params[3]) * scale.z);
verticies.add(vertex);
radius = Math.max(radius, vertex.length());
}
if (command.equals("center")) {
epicenter = new Vector3f(Float.parseFloat(params[1]), Float.parseFloat(params[2]), Float.parseFloat(params[3]));
} ...
| public static boolean canWeConnectToInternet() {
String s = "http://www.google.com/";
URL url = null;
boolean can = false;
URLConnection conection = null;
try {
url = new URL(s);
} catch (MalformedURLException e) {
System.out.println("This should never happend. Error in URL name. URL specified was:" + s + ".");
}
try {
conection = url.openConnection();
conection.connect();
can = true;
} catch (IOException e) {
can = false;
}
if (can) {
}
return can;
}
| 0.0 | | public static java.io.ByteArrayOutputStream getFileByteStream(URL _url) {
java.io.ByteArrayOutputStream buffer = new java.io.ByteArrayOutputStream();
try {
InputStream input = _url.openStream();
IOUtils.copy(input, buffer);
IOUtils.closeQuietly(input);
} catch (Exception err) {
throw new RuntimeException(err);
}
return buffer;
}
| public BufferedImage processUsingTemp(InputStream input, DjatokaDecodeParam params) throws DjatokaException {
File in;
try {
in = File.createTempFile("tmp", ".jp2");
FileOutputStream fos = new FileOutputStream(in);
in.deleteOnExit();
IOUtils.copyStream(input, fos);
} catch (IOException e) {
logger.error(e, e);
throw new DjatokaException(e);
}
BufferedImage bi = process(in.getAbsolutePath(), params);
if (in != null) in.delete();
return bi;
}
| 1.0 | * Loss: [CosineSimilarityLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosinesimilarityloss) with these parameters: ```json { "loss_fct": "torch.nn.modules.loss.MSELoss", "cos_score_transformation": "torch.nn.modules.linear.Identity" } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `per_device_train_batch_size`: 32 - `num_train_epochs`: 2 - `fp16`: True - `per_device_eval_batch_size`: 32 - `multi_dataset_batch_sampler`: round_robin #### All Hyperparameters
Click to expand - `per_device_train_batch_size`: 32 - `num_train_epochs`: 2 - `max_steps`: -1 - `learning_rate`: 5e-05 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: None - `warmup_steps`: 0 - `optim`: adamw_torch_fused - `optim_args`: None - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `optim_target_modules`: None - `gradient_accumulation_steps`: 1 - `average_tokens_across_devices`: True - `max_grad_norm`: 1 - `label_smoothing_factor`: 0.0 - `bf16`: False - `fp16`: True - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `use_liger_kernel`: False - `liger_kernel_config`: None - `use_cache`: False - `neftune_noise_alpha`: None - `torch_empty_cache_steps`: None - `auto_find_batch_size`: False - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `include_num_input_tokens_seen`: no - `log_level`: passive - `log_level_replica`: warning - `disable_tqdm`: False - `project`: huggingface - `trackio_space_id`: None - `trackio_bucket_id`: None - `trackio_static_space_id`: None - `per_device_eval_batch_size`: 32 - `prediction_loss_only`: True - `eval_on_start`: False - `eval_do_concat_batches`: True - `eval_use_gather_object`: False - `eval_accumulation_steps`: None - `include_for_metrics`: [] - `batch_eval_metrics`: False - `save_only_model`: False - `save_on_each_node`: False - `enable_jit_checkpoint`: False - `push_to_hub`: False - `hub_private_repo`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_always_push`: False - `hub_revision`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `restore_callback_states_from_checkpoint`: False - `full_determinism`: False - `seed`: 42 - `data_seed`: None - `use_cpu`: False - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `parallelism_config`: None - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `dataloader_prefetch_factor`: None - `dataloader_multiprocessing_context`: None - `dataloader_in_order`: True - `remove_unused_columns`: True - `label_names`: None - `train_sampling_strategy`: random - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `ddp_static_graph`: None - `ddp_backend`: None - `ddp_timeout`: 1800 - `fsdp`: None - `fsdp_config`: None - `deepspeed`: None - `debug`: [] - `skip_memory_metrics`: True - `do_predict`: False - `resume_from_checkpoint`: None - `local_rank`: -1 - `prompts`: None - `batch_sampler`: batch_sampler - `multi_dataset_batch_sampler`: round_robin - `router_mapping`: {} - `learning_rate_mapping`: {} - `warmup_ratio`: None
### Training Logs | Epoch | Step | Training Loss | bigclonebench-validation_spearman_cosine | |:------:|:----:|:-------------:|:----------------------------------------:| | 0.4984 | 156 | - | 0.8142 | | 0.9968 | 312 | - | 0.8325 | | 1.0 | 313 | - | 0.8325 | | 1.4952 | 468 | - | 0.8346 | | 1.5974 | 500 | 0.0901 | - | | 1.9936 | 624 | - | 0.8343 | | 2.0 | 626 | - | 0.8343 | ### Training Time - **Training**: 13.3 minutes ### Framework Versions - Python: 3.13.16 - Sentence Transformers: 5.7.0 - Transformers: 5.22.1 - PyTorch: 2.14.1+cu130 - Accelerate: 1.15.0 - Datasets: 4.8.5 - Tokenizers: 0.23.3 ## Additional Resources - [Training and Finetuning Embedding Models with Sentence Transformers](https://huggingface.co/blog/train-sentence-transformers): the end-to-end guide for training or finetuning Sentence Transformer models. - [Introduction to Matryoshka Embedding Models](https://huggingface.co/blog/matryoshka): variable-size embeddings that can be truncated with minimal quality loss. - [Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval](https://huggingface.co/blog/embedding-quantization): post-training compression of embedding vectors. - [Multimodal Embedding & Reranker Models with Sentence Transformers](https://huggingface.co/blog/multimodal-sentence-transformers): use text, image, audio, and video models through the same API. - [Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers](https://huggingface.co/blog/train-multimodal-sentence-transformers): train multimodal embedding models, with a Visual Document Retrieval walkthrough. ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ```