Upload 64 files

SVFT-main/LLM-Adapters/DATA_LICENSE

@@ -0,0 +1,183 @@
+Attribution License (ODC-By)
+PREAMBLE
+The Open Data Commons Attribution License is a license agreement intended to allow users to freely share, modify, and use this Database subject only to the attribution requirements set out in Section 4.
+
+Databases can contain a wide variety of types of content (images, audiovisual material, and sounds all in the same database, for example), and so this license only governs the rights over the Database, and not the contents of the Database individually. Licensors may therefore wish to use this license together with another license for the contents.
+
+Sometimes the contents of a database, or the database itself, can be covered by other rights not addressed here (such as private contracts, trademark over the name, or privacy rights / data protection rights over information in the contents), and so you are advised that you may have to consult other documents or clear other rights before doing activities not covered by this License.
+
+The Licensor (as defined below)
+
+and
+
+You (as defined below)
+
+agree as follows:
+
+1.0 DEFINITIONS OF CAPITALISED WORDS
+“Collective Database” – Means this Database in unmodified form as part of a collection of independent databases in themselves that together are assembled into a collective whole. A work that constitutes a Collective Database will not be considered a Derivative Database.
+
+“Convey” – As a verb, means Using the Database, a Derivative Database, or the Database as part of a Collective Database in any way that enables a Person to make or receive copies of the Database or a Derivative Database. Conveying does not include interaction with a user through a computer network, or creating and Using a Produced Work, where no transfer of a copy of the Database or a Derivative Database occurs.
+
+“Contents” – The contents of this Database, which includes the information, independent works, or other material collected into the Database. For example, the contents of the Database could be factual data or works such as images, audiovisual material, text, or sounds.
+
+“Database” – A collection of material (the Contents) arranged in a systematic or methodical way and individually accessible by electronic or other means offered under the terms of this License.
+
+“Database Directive” – Means Directive 96/9/EC of the European Parliament and of the Council of 11 March 1996 on the legal protection of databases, as amended or succeeded.
+
+“Database Right” – Means rights resulting from the Chapter III (“sui generis”) rights in the Database Directive (as amended and as transposed by member states), which includes the Extraction and Re-utilisation of the whole or a Substantial part of the Contents, as well as any similar rights available in the relevant jurisdiction under Section 10.4.
+
+“Derivative Database” – Means a database based upon the Database, and includes any translation, adaptation, arrangement, modification, or any other alteration of the Database or of a Substantial part of the Contents. This includes, but is not limited to, Extracting or Re-utilising the whole or a Substantial part of the Contents in a new Database.
+
+“Extraction” – Means the permanent or temporary transfer of all or a Substantial part of the Contents to another medium by any means or in any form.
+
+“License” – Means this license agreement and is both a license of rights such as copyright and Database Rights and an agreement in contract.
+
+“Licensor” – Means the Person that offers the Database under the terms of this License.
+
+“Person” – Means a natural or legal person or a body of persons corporate or incorporate.
+
+“Produced Work” – a work (such as an image, audiovisual material, text, or sounds) resulting from using the whole or a Substantial part of the Contents (via a search or other query) from this Database, a Derivative Database, or this Database as part of a Collective Database.
+
+“Publicly” – means to Persons other than You or under Your control by either more than 50% ownership or by the power to direct their activities (such as contracting with an independent consultant).
+
+“Re-utilisation” – means any form of making available to the public all or a Substantial part of the Contents by the distribution of copies, by renting, by online or other forms of transmission.
+
+“Substantial” – Means substantial in terms of quantity or quality or a combination of both. The repeated and systematic Extraction or Re-utilisation of insubstantial parts of the Contents may amount to the Extraction or Re-utilisation of a Substantial part of the Contents.
+
+“Use” – As a verb, means doing any act that is restricted by copyright or Database Rights whether in the original medium or any other; and includes without limitation distributing, copying, publicly performing, publicly displaying, and preparing derivative works of the Database, as well as modifying the Database as may be technically necessary to use it in a different mode or format.
+
+“You” – Means a Person exercising rights under this License who has not previously violated the terms of this License with respect to the Database, or who has received express permission from the Licensor to exercise rights under this License despite a previous violation.
+
+Words in the singular include the plural and vice versa.
+
+2.0 WHAT THIS LICENSE COVERS
+2.1. Legal effect of this document. This License is:
+
+a. A license of applicable copyright and neighbouring rights;
+
+b. A license of the Database Right; and
+
+c. An agreement in contract between You and the Licensor.
+
+2.2 Legal rights covered. This License covers the legal rights in the Database, including:
+
+a. Copyright. Any copyright or neighbouring rights in the Database. The copyright licensed includes any individual elements of the Database, but does not cover the copyright over the Contents independent of this Database. See Section 2.4 for details. Copyright law varies between jurisdictions, but is likely to cover: the Database model or schema, which is the structure, arrangement, and organisation of the Database, and can also include the Database tables and table indexes; the data entry and output sheets; and the Field names of Contents stored in the Database;
+
+b. Database Rights. Database Rights only extend to the Extraction and Re-utilisation of the whole or a Substantial part of the Contents. Database Rights can apply even when there is no copyright over the Database. Database Rights can also apply when the Contents are removed from the Database and are selected and arranged in a way that would not infringe any applicable copyright; and
+
+c. Contract. This is an agreement between You and the Licensor for access to the Database. In return you agree to certain conditions of use on this access as outlined in this License.
+
+2.3 Rights not covered.
+
+a. This License does not apply to computer programs used in the making or operation of the Database;
+
+b. This License does not cover any patents over the Contents or the Database; and
+
+c. This License does not cover any trademarks associated with the Database.
+
+2.4 Relationship to Contents in the Database. The individual items of the Contents contained in this Database may be covered by other rights, including copyright, patent, data protection, privacy, or personality rights, and this License does not cover any rights (other than Database Rights or in contract) in individual Contents contained in the Database.
+
+For example, if used on a Database of images (the Contents), this License would not apply to copyright over individual images, which could have their own separate licenses, or one single license covering all of the rights over the images.
+
+3.0 RIGHTS GRANTED
+3.1 Subject to the terms and conditions of this License, the Licensor grants to You a worldwide, royalty-free, non-exclusive, terminable (but only under Section 9) license to Use the Database for the duration of any applicable copyright and Database Rights. These rights explicitly include commercial use, and do not exclude any field of endeavour. To the extent possible in the relevant jurisdiction, these rights may be exercised in all media and formats whether now known or created in the future.
+
+The rights granted cover, for example:
+
+a. Extraction and Re-utilisation of the whole or a Substantial part of the Contents;
+
+b. Creation of Derivative Databases;
+
+c. Creation of Collective Databases;
+
+d. Creation of temporary or permanent reproductions by any means and in any form, in whole or in part, including of any Derivative Databases or as a part of Collective Databases; and
+
+e. Distribution, communication, display, lending, making available, or performance to the public by any means and in any form, in whole or in part, including of any Derivative Database or as a part of Collective Databases.
+
+3.2 Compulsory license schemes. For the avoidance of doubt:
+
+a. Non-waivable compulsory license schemes. In those jurisdictions in which the right to collect royalties through any statutory or compulsory licensing scheme cannot be waived, the Licensor reserves the exclusive right to collect such royalties for any exercise by You of the rights granted under this License;
+
+b. Waivable compulsory license schemes. In those jurisdictions in which the right to collect royalties through any statutory or compulsory licensing scheme can be waived, the Licensor waives the exclusive right to collect such royalties for any exercise by You of the rights granted under this License; and,
+
+c. Voluntary license schemes. The Licensor waives the right to collect royalties, whether individually or, in the event that the Licensor is a member of a collecting society that administers voluntary licensing schemes, via that society, from any exercise by You of the rights granted under this License.
+
+3.3 The right to release the Database under different terms, or to stop distributing or making available the Database, is reserved. Note that this Database may be multiple-licensed, and so You may have the choice of using alternative licenses for this Database. Subject to Section 10.4, all other rights not expressly granted by Licensor are reserved.
+
+4.0 CONDITIONS OF USE
+4.1 The rights granted in Section 3 above are expressly made subject to Your complying with the following conditions of use. These are important conditions of this License, and if You fail to follow them, You will be in material breach of its terms.
+
+4.2 Notices. If You Publicly Convey this Database, any Derivative Database, or the Database as part of a Collective Database, then You must:
+
+a. Do so only under the terms of this License;
+
+b. Include a copy of this License or its Uniform Resource Identifier (URI) with the Database or Derivative Database, including both in the Database or Derivative Database and in any relevant documentation;
+
+c. Keep intact any copyright or Database Right notices and notices that refer to this License; and
+
+d. If it is not possible to put the required notices in a particular file due to its structure, then You must include the notices in a location (such as a relevant directory) where users would be likely to look for it.
+
+4.3 Notice for using output (Contents). Creating and Using a Produced Work does not require the notice in Section 4.2. However, if you Publicly Use a Produced Work, You must include a notice associated with the Produced Work reasonably calculated to make any Person that uses, views, accesses, interacts with, or is otherwise exposed to the Produced Work aware that Content was obtained from the Database, Derivative Database, or the Database as part of a Collective Database, and that it is available under this License.
+
+a. Example notice. The following text will satisfy notice under Section 4.3:
+
+Contains information from DATABASE NAME which is made available
+under the ODC Attribution License.
+DATABASE NAME should be replaced with the name of the Database and a hyperlink to the location of the Database. “ODC Attribution License” should contain a hyperlink to the URI of the text of this License. If hyperlinks are not possible, You should include the plain text of the required URI’s with the above notice.
+
+4.4 Licensing of others. You may not sublicense the Database. Each time You communicate the Database, the whole or Substantial part of the Contents, or any Derivative Database to anyone else in any way, the Licensor offers to the recipient a license to the Database on the same terms and conditions as this License. You are not responsible for enforcing compliance by third parties with this License, but You may enforce any rights that You have over a Derivative Database. You are solely responsible for any modifications of a Derivative Database made by You or another Person at Your direction. You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License.
+
+5.0 MORAL RIGHTS
+5.1 Moral rights. This section covers moral rights, including any rights to be identified as the author of the Database or to object to treatment that would otherwise prejudice the author’s honour and reputation, or any other derogatory treatment:
+
+a. For jurisdictions allowing waiver of moral rights, Licensor waives all moral rights that Licensor may have in the Database to the fullest extent possible by the law of the relevant jurisdiction under Section 10.4;
+
+b. If waiver of moral rights under Section 5.1 a in the relevant jurisdiction is not possible, Licensor agrees not to assert any moral rights over the Database and waives all claims in moral rights to the fullest extent possible by the law of the relevant jurisdiction under Section 10.4; and
+
+c. For jurisdictions not allowing waiver or an agreement not to assert moral rights under Section 5.1 a and b, the author may retain their moral rights over certain aspects of the Database.
+
+Please note that some jurisdictions do not allow for the waiver of moral rights, and so moral rights may still subsist over the Database in some jurisdictions.
+
+6.0 FAIR DEALING, DATABASE EXCEPTIONS, AND OTHER RIGHTS NOT AFFECTED
+6.1 This License does not affect any rights that You or anyone else may independently have under any applicable law to make any use of this Database, including without limitation:
+
+a. Exceptions to the Database Right including: Extraction of Contents from non-electronic Databases for private purposes, Extraction for purposes of illustration for teaching or scientific research, and Extraction or Re-utilisation for public security or an administrative or judicial procedure.
+
+b. Fair dealing, fair use, or any other legally recognised limitation or exception to infringement of copyright or other applicable laws.
+
+6.2 This License does not affect any rights of lawful users to Extract and Re-utilise insubstantial parts of the Contents, evaluated quantitatively or qualitatively, for any purposes whatsoever, including creating a Derivative Database (subject to other rights over the Contents, see Section 2.4). The repeated and systematic Extraction or Re-utilisation of insubstantial parts of the Contents may however amount to the Extraction or Re-utilisation of a Substantial part of the Contents.
+
+7.0 WARRANTIES AND DISCLAIMER
+7.1 The Database is licensed by the Licensor “as is” and without any warranty of any kind, either express, implied, or arising by statute, custom, course of dealing, or trade usage. Licensor specifically disclaims any and all implied warranties or conditions of title, non-infringement, accuracy or completeness, the presence or absence of errors, fitness for a particular purpose, merchantability, or otherwise. Some jurisdictions do not allow the exclusion of implied warranties, so this exclusion may not apply to You.
+
+8.0 LIMITATION OF LIABILITY
+8.1 Subject to any liability that may not be excluded or limited by law, the Licensor is not liable for, and expressly excludes, all liability for loss or damage however and whenever caused to anyone by any use under this License, whether by You or by anyone else, and whether caused by any fault on the part of the Licensor or not. This exclusion of liability includes, but is not limited to, any special, incidental, consequential, punitive, or exemplary damages such as loss of revenue, data, anticipated profits, and lost business. This exclusion applies even if the Licensor has been advised of the possibility of such damages.
+
+8.2 If liability may not be excluded by law, it is limited to actual and direct financial loss to the extent it is caused by proved negligence on the part of the Licensor.
+
+9.0 TERMINATION OF YOUR RIGHTS UNDER THIS LICENSE
+9.1 Any breach by You of the terms and conditions of this License automatically terminates this License with immediate effect and without notice to You. For the avoidance of doubt, Persons who have received the Database, the whole or a Substantial part of the Contents, Derivative Databases, or the Database as part of a Collective Database from You under this License will not have their licenses terminated provided their use is in full compliance with this License or a license granted under Section 4.8 of this License. Sections 1, 2, 7, 8, 9 and 10 will survive any termination of this License.
+
+9.2 If You are not in breach of the terms of this License, the Licensor will not terminate Your rights under it.
+
+9.3 Unless terminated under Section 9.1, this License is granted to You for the duration of applicable rights in the Database.
+
+9.4 Reinstatement of rights. If you cease any breach of the terms and conditions of this License, then your full rights under this License will be reinstated:
+
+a. Provisionally and subject to permanent termination until the 60th day after cessation of breach;
+
+b. Permanently on the 60th day after cessation of breach unless otherwise reasonably notified by the Licensor; or
+
+c. Permanently if reasonably notified by the Licensor of the violation, this is the first time You have received notice of violation of this License from the Licensor, and You cure the violation prior to 30 days after your receipt of the notice.
+
+9.5 Notwithstanding the above, Licensor reserves the right to release the Database under different license terms or to stop distributing or making available the Database. Releasing the Database under different license terms or stopping the distribution of the Database will not withdraw this License (or any other license that has been, or is required to be, granted under the terms of this License), and this License will continue in full force and effect unless terminated as stated above.
+
+10.0 GENERAL
+10.1 If any provision of this License is held to be invalid or unenforceable, that must not affect the validity or enforceability of the remainder of the terms and conditions of this License and each remaining provision of this License shall be valid and enforced to the fullest extent permitted by law.
+
+10.2 This License is the entire agreement between the parties with respect to the rights granted here over the Database. It replaces any earlier understandings, agreements or representations with respect to the Database.
+
+10.3 If You are in breach of the terms of this License, You will not be entitled to rely on the terms of this License or to complain of any breach by the Licensor.
+
+10.4 Choice of law. This License takes effect in and will be governed by the laws of the relevant jurisdiction in which the License terms are sought to be enforced. If the standard suite of rights granted under applicable copyright law and Database Rights in the relevant jurisdiction includes additional rights not granted under this License, these additional rights are granted in this License in order to meet the terms of this License.

SVFT-main/LLM-Adapters/LICENSE

@@ -0,0 +1,201 @@
+                                Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2023 Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

SVFT-main/LLM-Adapters/README.md

@@ -0,0 +1,267 @@
+<!---
+Copyright 2023 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+<h1 align="center"> 
+<img src="picture.jpg" width="73" height="114">
+<p> LLM-Adapters</p>
+</h1>
+
+<h3 align="center">
+    <p>LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models </p>
+</h3>
+LLM-Adapters is an easy-to-use framework that integrates various adapters into LLMs and can execute adapter-based PEFT methods of LLMs for different tasks. LLM-Adapter is an extension of HuggingFace's PEFT library, many thanks for their amazing work! Please find our paper at this link: https://arxiv.org/abs/2304.01933.
+
+The framework includes state-of-the-art open-access LLMs: LLaMa, OPT, BLOOM, and GPT-J, as well as widely used adapters such as Bottleneck adapters, Parallel adapters, and LoRA.
+
+Supported Adapters:
+
+1. LoRA: [LORA: LOW-RANK ADAPTATION OF LARGE LANGUAGE MODELS](https://arxiv.org/pdf/2106.09685.pdf)
+2. AdapterH: [Parameter-Efficient Transfer Learning for NLP](https://arxiv.org/pdf/1902.00751.pdf)
+3. AdapterP: [GMAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer](https://arxiv.org/pdf/2005.00052.pdf)
+4. Parallel: [TOWARDS A UNIFIED VIEW OF PARAMETER-EFFICIENT TRANSFER LEARNING](https://arxiv.org/pdf/2110.04366.pdf)
+5. Prefix Tuning: [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://aclanthology.org/2021.acl-long.353/), [P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks](https://arxiv.org/pdf/2110.07602.pdf)
+6. P-Tuning: [GPT Understands, Too](https://arxiv.org/pdf/2103.10385.pdf)
+7. Prompt Tuning: [The Power of Scale for Parameter-Efficient Prompt Tuning](https://arxiv.org/pdf/2104.08691.pdf) 
+
+## Latest News 🔥🔥
+
+* [2023-08-10] LLM-Adapters has been accepted by EMNLP 2023.
+* [2023-07-16] we released commonsense170k dataset and the  The LLaMA-13B-Parallel model outformances ChatGPT on 8 commonsense benchmarks.
+* [2023-04-21] We released math10k dataset and the [LLaMA-13B adapter checkpoints](https://drive.google.com/file/d/1NqUv-Hn_mAkGXsUOqpJKmPKW5Gp8mRlO/view?usp=sharing). The LLaMA-13B-Parallel model achieves **91%** of GPT-3.5 performance!
+* [2023-04-10] We can support GPT-Neo and ChatGLM now!
+* [2023-04-04] [Release code and dataset](https://github.com/AGI-Edgerunners/LLM-Adapters)
+
+## Special Announcement
+The `math_10k.json` data is collected with the training sets of GSM8K, MAWPS, and AQuA(1000 examples). However, MAWPS consists of AddSub, MultiArith, SingleOp, SingleEq, SimulEq-S, SimulEq-L. Thus, we can't utilize MultiArith, AddSub, and SingleEq as evaluation benchmarks with models trained with `math_10k.json`. We evaluate the PEFT methods on the MAWPS test set instead, and the result table has been updated (The findings in the paper are consistent). Furthermore, two variations of `math_10k.json` have been uploaded, `math_7K.json` where the MAWPS samples have been deleted, and `math_14k.json` where the MAWPS samples have been deleted as well and we combine ChatGPT and GPT-4 rationales. Sincerely apologize for any inconvenience!
+
+## Setup
+
+1. Install dependencies
+```bash
+pip install -r requirements.txt
+```
+
+2. Set environment variables, or modify the files referencing `BASE_MODEL`:
+
+```bash
+# Files referencing `BASE_MODEL`
+# export_hf_checkpoint.py
+# export_state_dict_checkpoint.py
+
+export BASE_MODEL=yahma/llama-7b-hf
+```
+
+Both `finetune.py` and `generate.py` use `--base_model` flag as shown further below.
+
+3. If bitsandbytes doesn't work, [install it from source.](https://github.com/TimDettmers/bitsandbytes/blob/main/compile_from_source.md) Windows users can follow [these instructions](https://github.com/tloen/alpaca-lora/issues/17).
+
+## Training(finetune.py)
+
+This file contains some code related to prompt construction and tokenization.In this file, specify different adapters and different sets of data, so that different models can be trained. 
+
+Example usage for multiple GPUs:
+
+```bash
+WORLD_SIZE=2 CUDA_VISIBLE_DEVICES=0,1 torchrun --nproc_per_node=2 --master_port=3192 finetune.py \
+  --base_model 'yahma/llama-7b-hf' \
+  --data_path 'math_10k.json' \
+  --output_dir './trained_models/llama-lora' \
+  --batch_size 16 \
+  --micro_batch_size 4 \
+  --num_epochs 3 \
+  --learning_rate 3e-4 \
+  --cutoff_len 256 \
+  --val_set_size 120 \
+  --adapter_name lora
+```
+
+The `math_10k.json` data is collected with the training sets of GSM8K, MAWPS, and AQuA(1000 examples). `yahma/llama-7b-hf` is a base model, LLaMa-7B. Add `lora` adapter to this model.
+
+Example usage for Single GPUs:
+
+```bash
+CUDA_VISIBLE_DEVICES=0 python finetune.py \
+  --base_model 'yahma/llama-7b-hf' \
+  --data_path 'math_10k.json' \
+  --output_dir './trained_models/llama-lora' \
+  --batch_size 16 \
+  --micro_batch_size 4 \
+  --num_epochs 3 \
+  --learning_rate 3e-4 \
+  --cutoff_len 256 \
+  --val_set_size 120 \
+  --adapter_name lora
+```
+
+Moreover, you can use `--use_gradient_checkpointing` to save more GPU memory, but it will increase the training time.
+
+To use the AdapterH, just add the following arguments:
+
+```bash
+--adapter_name bottleneck # use the bottleneck adapter, refers to AdapterH in the result table
+```
+
+To use the AdapterP, just add the following arguments:
+
+```bash
+--adapter_name bottleneck 
+--use_adapterp  # use the AdapterP, refers to AdapterP in the result table
+```
+
+To use parallel adapter, just add the following arguments:
+
+```bash
+--adapter_name bottleneck
+--use_parallel_adapter
+```
+
+Note that, In order to facilitate INT8 training of large models with parallel adapters, we have adopted a technique whereby the parallel adapter layers are incorporated into multi-head attention layers and MLP layers, in parallel with Linear layers. It is different from [Hu et al. (2021)](https://arxiv.org/pdf/2106.09685.pdf). 
+
+## Inference (generate.py)
+
+This file reads the foundation model from the Hugging Face model hub and the LoRA weights from `'./trained_models/llama-lora'` , and runs a Gradio interface for inference on a specified input. Users should treat this as example code for the use of the model, and modify it as needed.
+Example usage:
+
+```bash
+CUDA_VISIBLE_DEVICES=0 torchrun generate.py \
+    --base_model 'yahma/llama-7b-hf' \
+    --lora_weights './trained_models/llama-lora'
+```
+
+## Evaluation (evaluate.py)
+
+To evaluate the performance of the finetuned model on the Arithmetic Reasoning tasks, you can use the following command:
+
+```bash
+CUDA_VISIBLE_DEVICES=0 python evaluate.py 
+    --model LLaMA-7B \ #specify the base model
+    --adapter LoRA \   #specify the adapter name ["LoRA", "AdapterH", "AdapterP", "Parallel"， "Scaled_Parallel""]
+    --dataset SVAMP \  #specify the test dataset
+    --base_model 'yahma/llama-7b-hf' \
+    --lora_weights './trained_models/llama-lora'
+```
+
+<!-- ## Resource Consumption
+
+There is a table of resouce needed for different adapters, which contains Trainable Parameters, GPU RAM Usage, and Fine-tuning Time on the Arithmetic Reasoning dataset `math_10k.json`
+
+Hyper-parameter setting: num_epochs=3, lora_r=8, lora_alpha=16, bottleneck_size=256
+
+Models: LLaMA-13B, LLaMA-7B, BLOOM-6.7B, GPT-j-6B
+Dataset: 3.2K math word problems
+
+Hardware: 2*3090 GPUs
+
+| Model                 | Trainable Parameters | GPU RAM Usage | Fine-tuning Time |
+|-----------------------|----------------------|---------------|------------------|
+| LLaMA-7B-LoRA         | 4.2M                 | 18GB          |     4h           | 
+| LLaMA-7B-AdapterH     | 200M                 | 22GB          |     4h           | 
+| LLaMA-7B-AdapterP     | 200M                 | 22GB          |     4h           | 
+| LLaMA-7B-Parallel     | 200M                 | 22GB          |     4h           |  -->
+
+
+## Finetune Result
+There are the finetune results in different models with 4 math reasoning datasets, which contains GSM8K, AQuA, SVAMP, and MAWPS. In this table, we use the optimal configuration and placement of Prefix-Tuning, Series Adapter, LoRA, and Parallel Adapter according to the empirical study in our [paper](https://aclanthology.org/2023.emnlp-main.319/).
+
+| Model                 | GSM8K  | AQuA   |   MAWPS  |  SVAMP | Average |
+|-----------------------|--------|--------|----------|--------|---------|
+| GPT-3.5               |**56.4**|**38.9**| **87.4** |**69.9**|**63.2** |
+| BLOOMz-7B-Prefix	    | 13.8   |  12.5  |   47.5   |  24.1  |  24.5   |
+| BLOOMz-7B-Series	    | 14.3   |  20.5  |   62.2   |  38.1  |  33.8   |
+| BLOOMz-7B-Parallel	| 18.5   |  18.9  |   70.6   |  36.4  |  36.1   |
+| BLOOMz-7B-LoRA	    | 17.4	 |  21.3  |   70.2   |  41.0  |  37.5   |
+| GPT-j-6B-Prefix	    | 16.0	 |  14.7  |   59.2   |  31.0  |  30.2   |
+| GPT-j-6B-Series	    | 19.5	 |  15.0  |   80.3   |  43.6  |  39.6   |
+| GPT-j-6B-Parallel	    | 18.9	 |  17.9  |   78.2   |  41.1  |  39.0   |
+| GPT-j-6B-LoRA	        | 23.0	 |  16.1  |   79.4   |  46.0  |  41.1   |
+| LLaMA-7B-Prefix	    | 24.4	 |  14.2  |   63.4   |  38.1  |  35.0   |
+| LLaMA-7B-Series	    | 33.3	 |  15.0  |   77.7   |  52.3  |  44.6   |
+| LLaMA-7B-Parallel	    | 35.3	 |  18.1  |   82.4   |  49.6  |  46.4   |
+| LLaMA-7B-LoRA	        | 37.5	 |  18.9  |   79.0   |  52.1  |  46.9   |
+| LLaMA-13B-Prefix	    | 31.1	 |  15.7  |   66.8   |  41.4  |  38.8   |
+| LLaMA-13B-Series	    | 44.0	 |  22.0  |   78.6   |  50.8  |  48.9   |
+| LLaMA-13B-Parallel	| 43.3	 |  20.5  |   81.1   |  55.7  |  50.2   |
+| LLaMA-13B-LoRA	    | 47.5	 |  18.5  |   83.6   |  54.6  |  51.1   |
+
+
+There are the finetune results in different models with eight commonsense reasoning datasets.
+
+| Model                 |  BoolQ  |  PIQA  |  SIQA  |  HellaSwag  |  WinoGrande  |  ARC-e  |  ARC-c  |  OBQA  |  Average  |
+|-----------------------|---------|--------|--------|-------------|--------------|---------|---------|--------|-----------|
+| ChatGPT               | **73.1**|**85.4**|  68.5  |  78.5       |  66.1        |**89.8** |**79.9** |  74.8  |  77.0     |
+| BLOOMz-7B-Prefix	    |   45.6  |  53.7  |  46.3  |  26.7       |  49.5        |  52.1   |  39.7   |  44.3  |  44.7     |
+| BLOOMz-7B-Series	    |   65.4  |  70.4  |  73.6  |  53.4       |  69.3        |  72.3   |  55.9   |  68.0  |  66.0     |
+| BLOOMz-7B-Parallel	  |   64.1  |  71.5  |  72.1  |  52.9       |  67.0        |  70.5   |  54.7   |  69.6  |  65.3     |
+| BLOOMz-7B-LoRA	      |   65.9  |  75.3  |  74.5  |  57.3       |  72.5        |  74.6   |  57.8   |  73.4  |  68.9     |
+| GPT-j-6B-Prefix	      |   63.1  |  66.9  |  68.7  |  34.4       |  64.5        |  64.4   |  46.8   |  59.0  |  58.5     |
+| GPT-j-6B-Series	      |   62.1  |  63.5  |  72.3  |  30.6       |  68.0        |  63.9   |  48.1   |  63.8  |  59.0     |
+| GPT-j-6B-Parallel	    |   62.2  |  69.7  |  70.0  |  41.7       |  65.0        |  60.2   |  44.6   |  58.2  |  59.0     |
+| GPT-j-6B-LoRA	        |   62.4  |  68.6  |  49.5  |  43.1       |  57.3        |  43.4   |  31.0   |  46.6  |  50.2     |
+| LLaMA-7B-Prefix	      |   64.3  |  76.8  |  73.9  |  42.1       |  72.1        |  72.9   |  54.0   |  60.6  |  64.6     |
+| LLaMA-7B-Series	      |   63.0  |  79.2  |  76.3  |  67.9       |  75.7        |  74.5   |  57.1   |  72.4  |  70.8     |
+| LLaMA-7B-Parallel	    |   67.9  |  76.4  |  78.8  |  69.8       |  78.9        |  73.7   |  57.3   |  75.2  |  72.3     |
+| LLaMA-7B-LoRA	        |   68.9  |  80.7  |  77.4  |  78.1       |  78.8        |  77.8   |  61.3   |  74.8  |  74.7     |
+| LLaMA-13B-Prefix	    |   65.3  |  75.4  |  72.1  |  55.2       |  68.6        |  79.5   |  62.9   |  68.0  |  68.4     |
+| LLaMA-13B-Series	    |   71.8  |  83.0  |  79.2  |  88.1       |  82.4        |  82.5   |  67.3   |  81.8  |  79.5     |
+| LLaMA-13B-Parallel	  |   72.5  |  84.8  |  79.8  |**92.1**     |**84.7**      |  84.2   |  71.2   |**82.4**|**81.5**   |
+| LLaMA-13B-LoRA	      |   72.1  |  83.5  |**80.5**|  90.5       |  83.7        |  82.8   |  68.3   |**82.4**|  80.5     |
+
+
+### Adapter support matrix
+This metrix shows whether different models can use LoRA,AdapterH,AdapterP,Parallel and Scaled Parallel adapters.
+
+| Adapter      | LoRA | AdapterH | AdapterP | Parallel| Prefix Tuning	|P-Tuning|Prompt Tuning|
+|--------------|-------|-------|----------|-------|-------|-------|-------|
+| LLaMA        | ✅     | ✅     | ✅        |✅     | ✅     | ✅     | ✅     |
+| BLOOM        | ✅     | ✅     | ✅        |✅     | ✅     | ✅     | ✅     | 
+| GPT-J        | ✅     | ✅     | ✅        |✅     | ✅     | ✅     | ✅     |
+| OPT          | ✅     | ✅     | ✅        |✅     | ✅     | ✅     | ✅     |
+| GPT-2        | ✅     | 🔧Developing | 🔧Developing|🔧Developing | ✅     | ✅     | ✅     | 
+| GPT-Neo      | ✅     | ✅     | ✅        | ✅    | ✅     | ✅     | ✅     | 
+| GPT-NeoX-20B | ✅     | 🔧Developing | 🔧Developing|🔧Developing | ✅     | ✅     | ✅     |
+| ChatGLM      | ✅     | ✅     | ✅        |✅     | ✅     | ✅     | ✅     | 
+
+
+### TODO List
+- [x] Add AdapterH
+- [x] Add AdapterP
+- [x] Add Parallel Adapter
+- [ ] Support More LLMs
+- [ ] Support Multiple Adapter
+- [ ] Support Adapter Composition
+- [ ] Support Adapter Fusion
+
+
+## :star: Star History
+
+[![Star History Chart](https://api.star-history.com/svg?repos=AGI-Edgerunners/LLM-Adapters&type=Date)](https://star-history.com/#AGI-Edgerunners/LLM-Adapters&Date)
+
+## Citing <img src="picture.jpg" width="14px" height="14px"> LLM-Adapter
+
+If you use <img src="picture.jpg" width="14px" height="14px"> LLM-Adapters in your publication, please cite it by using the following BibTeX entry.
+
+```bibtex
+@article{hu2023llm,
+  title={LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models},
+  author={Hu, Zhiqiang and Lan, Yihuai and Wang, Lei and Xu, Wanyu and Lim, Ee-Peng and Lee, Roy Ka-Wei and Bing, Lidong and Poria, Soujanya},
+  journal={arXiv preprint arXiv:2304.01933},
+  year={2023}
+}
+```
+
+## Acknowledgement
+
+This repo benefits from [PEFT](https://github.com/huggingface/peft), [Adapter-Transformer](https://github.com/adapter-hub/adapter-transformers), [Alpaca-lora](https://github.com/tloen/alpaca-lora). Thanks for their wonderful works. Additionally, we thank DONG Shan and [dream.ai](https://dream.ai/create) for the exceptional logo design, which has added immense value to our project.

SVFT-main/LLM-Adapters/commonsense_evaluate.py

@@ -0,0 +1,300 @@
+import copy
+import json
+import os
+import re
+import sys
+import argparse
+
+import fire
+
+import torch
+
+sys.path.append(os.path.join(os.getcwd(), "peft/src/"))
+from peft import PeftModel
+from tqdm import tqdm
+from transformers import GenerationConfig, LlamaForCausalLM, LlamaTokenizer, AutoModelForCausalLM, AutoTokenizer
+
+if torch.cuda.is_available():
+    device = "cuda"
+else:
+    device = "cpu"
+
+try:
+    if torch.backends.mps.is_available():
+        device = "mps"
+except:  # noqa: E722
+    pass
+
+
+def main(
+        load_8bit: bool = False,
+        base_model: str = "",
+        lora_weights: str = "tloen/alpaca-lora-7b",
+        share_gradio: bool = False,
+):
+    args = parse_args()
+
+    def evaluate(
+            instructions,
+            input=None,
+            temperature=0.1,
+            top_p=0.75,
+            top_k=40,
+            num_beams=4,
+            max_new_tokens=32,
+            **kwargs,
+    ):
+        prompts = [generate_prompt(instruction, input) for instruction in instructions]
+        inputs = tokenizer(prompts, return_tensors="pt", padding=True)
+        input_ids = inputs["input_ids"].to(device)
+        generation_config = GenerationConfig(
+            temperature=temperature,
+            top_p=top_p,
+            top_k=top_k,
+            num_beams=num_beams,
+            **kwargs,
+        )
+        with torch.no_grad():
+            generation_output = model.generate(
+                input_ids=input_ids,
+                generation_config=generation_config,
+                return_dict_in_generate=True,
+                output_scores=True,
+                max_new_tokens=max_new_tokens,
+            )
+        s = generation_output.sequences
+        outputs = tokenizer.batch_decode(s, skip_special_tokens=True)
+        outputs = [o.split("### Response:")[1].strip() for o in outputs]
+        print(outputs)
+        return outputs
+
+    save_file = f'experiment/{args.model}-{args.adapter}-{args.dataset}.json'
+    create_dir('experiment/')
+
+    dataset = load_data(args)
+    batches = create_batch(dataset, args.batch_size)
+    tokenizer, model = load_model(args)
+    total = len(batches)
+    correct = 0
+    current = 0
+    output_data = []
+    pbar = tqdm(total=total)
+    for idx, batch in enumerate(batches):
+        current += len(batch)
+        instructions = [data.get('instruction') for data in batch]
+
+        outputs = evaluate(instructions)
+
+        for data, output in zip(batch, outputs):
+            label = data.get('answer')
+            flag = False
+            predict = extract_answer(args, output)
+            if label == predict:
+                correct += 1
+                flag = True
+            new_data = copy.deepcopy(data)
+            new_data['output_pred'] = output
+            new_data['pred'] = predict
+            new_data['flag'] = flag
+            output_data.append(new_data)
+            print(data["instruction"])
+            print(output)
+            print('prediction:', predict)
+            print('label:', label)
+        print('---------------')
+        print(f'\rtest:{idx + 1}/{total} | accuracy {correct}  {correct / current}')
+        print('---------------')
+        with open(save_file, 'w+') as f:
+            json.dump(output_data, f, indent=4)
+        pbar.update(1)
+    pbar.close()
+    print('\n')
+    print('test finished')
+
+
+def create_dir(dir_path):
+    if not os.path.exists(dir_path):
+        os.mkdir(dir_path)
+    return
+
+
+def generate_prompt(instruction, input=None):
+    if input:
+        return f"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
+
+                ### Instruction:
+                {instruction}
+
+                ### Input:
+                {input}
+
+                ### Response:
+                """  # noqa: E501
+    else:
+        return f"""Below is an instruction that describes a task. Write a response that appropriately completes the request. 
+
+                ### Instruction:
+                {instruction}
+
+                ### Response:
+                """  # noqa: E501
+
+
+def load_data(args) -> list:
+    """
+    read data from dataset file
+    Args:
+        args:
+
+    Returns:
+
+    """
+    file_path = f'dataset/{args.dataset}/test.json'
+    if not os.path.exists(file_path):
+        raise FileNotFoundError(f"can not find dataset file : {file_path}")
+    json_data = json.load(open(file_path, 'r'))
+    return json_data
+
+def create_batch(dataset, batch_size):
+    batches = []
+    num_batch = len(dataset)//batch_size if len(dataset) % batch_size == 0 else len(dataset)//batch_size + 1
+    for i in range(num_batch):
+        batch = dataset[i*batch_size: min((i+1)*batch_size, len(dataset))]
+        batches.append(batch)
+    return batches
+
+
+def parse_args():
+    parser = argparse.ArgumentParser()
+    parser.add_argument('--dataset', choices=["boolq", "piqa", "social_i_qa", "hellaswag", "winogrande", "ARC-Challenge", "ARC-Easy", "openbookqa"],
+                        required=True)
+    parser.add_argument('--model', choices=['LLaMA-7B', "LLaMA-13B",'BLOOM-7B', 'GPT-j-6B'], required=True)
+    parser.add_argument('--adapter', choices=['LoRA', 'AdapterP', 'AdapterH', 'Parallel'],
+                        required=True)
+    parser.add_argument('--base_model', required=True)
+    parser.add_argument('--lora_weights', required=True)
+    parser.add_argument('--batch_size', type=int, required=True)
+    parser.add_argument('--load_8bit', action='store_true', default=False)
+
+    return parser.parse_args()
+
+
+def load_model(args) -> tuple:
+    """
+    load tuned model
+    Args:
+        args:
+
+    Returns:
+        tuple(tokenizer, model)
+    """
+    base_model = args.base_model
+    if not base_model:
+        raise ValueError(f'can not find base model name by the value: {args.model}')
+    lora_weights = args.lora_weights
+    if not lora_weights:
+        raise ValueError(f'can not find lora weight, the value is: {lora_weights}')
+
+    load_8bit = args.load_8bit
+    if "LLaMA" in args.model:
+        tokenizer = LlamaTokenizer.from_pretrained(base_model)
+    else:
+        tokenizer = AutoTokenizer.from_pretrained(base_model)
+    tokenizer.padding_side = "left"
+    tokenizer.pad_token_id = (
+        0  # unk. we want this to be different from the eos token
+    )
+    if device == "cuda":
+        model = AutoModelForCausalLM.from_pretrained(
+            base_model,
+            load_in_8bit=load_8bit,
+            torch_dtype=torch.float16,
+            device_map="auto",
+            trust_remote_code=True,
+        ) # fix zwq
+        model = PeftModel.from_pretrained(
+            model,
+            lora_weights,
+            torch_dtype=torch.float16,
+            device_map={"":0}
+        )
+    elif device == "mps":
+        model = AutoModelForCausalLM.from_pretrained(
+            base_model,
+            device_map={"": device},
+            torch_dtype=torch.float16,
+        )
+        model = PeftModel.from_pretrained(
+            model,
+            lora_weights,
+            device_map={"": device},
+            torch_dtype=torch.float16,
+        )
+    else:
+        model = AutoModelForCausalLM.from_pretrained(
+            base_model, device_map={"": device}, low_cpu_mem_usage=True
+        )
+        model = PeftModel.from_pretrained(
+            model,
+            lora_weights,
+            device_map={"": device},
+        )
+
+        # unwind broken decapoda-research config
+        model.config.pad_token_id = tokenizer.pad_token_id = 0  # unk
+        model.config.bos_token_id = 1
+        model.config.eos_token_id = 2
+
+        if not load_8bit:
+            model.half()  # seems to fix bugs for some users.
+
+        model.eval()
+        if torch.__version__ >= "2" and sys.platform != "win32":
+            model = torch.compile(model)
+
+    return tokenizer, model
+
+
+def load_instruction(args) -> str:
+    instruction = ''
+    if not instruction:
+        raise ValueError('instruct not initialized')
+    return instruction
+
+
+def extract_answer(args, sentence: str) -> float:
+    dataset = args.dataset
+    if dataset == 'boolq':
+        sentence_ = sentence.strip()
+        pred_answers = re.findall(r'true|false', sentence_)
+        if not pred_answers:
+            return ""
+        return pred_answers[0]
+    elif dataset == 'piqa':
+        sentence_ = sentence.strip()
+        pred_answers = re.findall(r'solution1|solution2', sentence_)
+        if not pred_answers:
+            return ""
+        return pred_answers[0]
+    elif dataset in ['social_i_qa', 'ARC-Challenge', 'ARC-Easy', 'openbookqa']:
+        sentence_ = sentence.strip()
+        pred_answers = re.findall(r'answer1|answer2|answer3|answer4|answer5', sentence_)
+        if not pred_answers:
+            return ""
+        return pred_answers[0]
+    elif dataset == 'hellaswag':
+        sentence_ = sentence.strip()
+        pred_answers = re.findall(r'ending1|ending2|ending3|ending4', sentence_)
+        if not pred_answers:
+            return ""
+        return pred_answers[0]
+    elif dataset == 'winogrande':
+        sentence_ = sentence.strip()
+        pred_answers = re.findall(r'option1|option2', sentence_)
+        if not pred_answers:
+            return ""
+        return pred_answers[0]
+
+
+if __name__ == "__main__":
+    main()

SVFT-main/LLM-Adapters/evaluate.py

@@ -0,0 +1,302 @@
+import copy
+import json
+import os
+import re
+import sys
+import argparse
+
+import fire
+
+import torch
+
+sys.path.append(os.path.join(os.getcwd(), "peft/src/"))
+from peft import PeftModel
+from tqdm import tqdm
+from transformers import GenerationConfig, LlamaForCausalLM, LlamaTokenizer, AutoModelForCausalLM, AutoTokenizer
+
+if torch.cuda.is_available():
+    device = "cuda"
+else:
+    device = "cpu"
+
+try:
+    if torch.backends.mps.is_available():
+        device = "mps"
+except:  # noqa: E722
+    pass
+
+
+def main(
+        load_8bit: bool = False,
+        base_model: str = "",
+        lora_weights: str = "tloen/alpaca-lora-7b",
+        share_gradio: bool = False,
+):
+    args = parse_args()
+
+    def evaluate(
+            instruction,
+            input=None,
+            temperature=0.1,
+            top_p=0.75,
+            top_k=40,
+            num_beams=4,
+            max_new_tokens=256,
+            **kwargs,
+    ):
+        prompt = generate_prompt(instruction, input)
+        inputs = tokenizer(prompt, return_tensors="pt")
+        input_ids = inputs["input_ids"].to(device)
+        generation_config = GenerationConfig(
+            temperature=temperature,
+            top_p=top_p,
+            top_k=top_k,
+            num_beams=num_beams,
+            **kwargs,
+        )
+        with torch.no_grad():
+            generation_output = model.generate(
+                input_ids=input_ids,
+                generation_config=generation_config,
+                return_dict_in_generate=True,
+                output_scores=True,
+                max_new_tokens=max_new_tokens,
+                use_cache=False,
+            )
+        s = generation_output.sequences[0]
+        output = tokenizer.decode(s)
+        return output.split("### Response:")[1].strip()
+
+    """
+    # testing code for readme
+    for instruction in [
+        "Tell me about alpacas.",
+        "Tell me about the president of Mexico in 2019.",
+        "Tell me about the king of France in 2019.",
+        "List all Canadian provinces in alphabetical order.",
+        "Write a Python program that prints the first 10 Fibonacci numbers.",
+        "Write a program that prints the numbers from 1 to 100. But for multiples of three print 'Fizz' instead of the number and for the multiples of five print 'Buzz'. For numbers which are multiples of both three and five print 'FizzBuzz'.",  # noqa: E501
+        "Tell me five words that rhyme with 'shock'.",
+        "Translate the sentence 'I have no mouth but I must scream' into Spanish.",
+        "Count up from 1 to 500.",
+    ]:
+        print("Instruction:", instruction)
+        print("Response:", evaluate(instruction))
+        print()
+    """
+    save_file = f'experiment/{args.model}-{args.adapter}-{args.dataset}.json'
+    create_dir('experiment/')
+
+    dataset = load_data(args)
+    tokenizer, model = load_model(args)
+    total = len(dataset)
+    correct = 0
+    miss = 0.001
+    output_data = []
+    pbar = tqdm(total=total)
+    for idx, data in enumerate(dataset):
+        instruction = data.get('instruction')
+
+        outputs = evaluate(instruction)
+        label = data.get('answer')
+        flag = False
+        if args.dataset.lower() in ['aqua']:
+            predict = extract_answer_letter(args, outputs)
+            if label == predict:
+                correct += 1
+                flag = True
+        else:
+            if isinstance(label, str):
+                label = float(label)
+            predict = extract_answer_number(args, outputs)
+            if abs(label - predict) <= miss:
+                correct += 1
+                flag = True
+        new_data = copy.deepcopy(data)
+        new_data['output_pred'] = outputs
+        new_data['pred'] = predict
+        new_data['flag'] = flag
+        output_data.append(new_data)
+        print(' ')
+        print('---------------')
+        print(outputs)
+        print('prediction:', predict)
+        print('label:', label)
+        print('---------------')
+        print(f'\rtest:{idx + 1}/{total} | accuracy {correct}  {correct / (idx + 1)}')
+        with open(save_file, 'w+') as f:
+            json.dump(output_data, f, indent=4)
+        pbar.update(1)
+    pbar.close()
+    print('\n')
+    print('test finished')
+
+
+def create_dir(dir_path):
+    if not os.path.exists(dir_path):
+        os.mkdir(dir_path)
+    return
+
+
+def generate_prompt(instruction, input=None):
+    if input:
+        return f"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
+
+                ### Instruction:
+                {instruction}
+
+                ### Input:
+                {input}
+
+                ### Response:
+                """  # noqa: E501
+    else:
+        return f"""Below is an instruction that describes a task. Write a response that appropriately completes the request. 
+
+                ### Instruction:
+                {instruction}
+
+                ### Response:
+                """  # noqa: E501
+
+
+def load_data(args) -> list:
+    """
+    read data from dataset file
+    Args:
+        args:
+
+    Returns:
+
+    """
+    file_path = f'dataset/{args.dataset}/test.json'
+    if not os.path.exists(file_path):
+        raise FileNotFoundError(f"can not find dataset file : {file_path}")
+    json_data = json.load(open(file_path, 'r'))
+    return json_data
+
+
+def parse_args():
+    parser = argparse.ArgumentParser()
+    parser.add_argument('--dataset', choices=['AddSub', 'MultiArith', 'SingleEq', 'gsm8k', 'AQuA', 'SVAMP'],
+                        required=True)
+    parser.add_argument('--model', choices=['LLaMA-7B', 'BLOOM-7B', 'GPT-j-6B'], required=True)
+    parser.add_argument('--adapter', choices=['LoRA', 'AdapterP', 'AdapterH', 'Parallel', 'Prefix'],
+                        required=True)
+    parser.add_argument('--base_model', required=True)
+    parser.add_argument('--lora_weights', required=True)
+    parser.add_argument('--load_8bit', action='store_true', default=False)
+
+    return parser.parse_args()
+
+
+def load_model(args) -> tuple:
+    """
+    load tuned model
+    Args:
+        args:
+
+    Returns:
+        tuple(tokenizer, model)
+    """
+    base_model = args.base_model
+    if not base_model:
+        raise ValueError(f'can not find base model name by the value: {args.model}')
+    lora_weights = args.lora_weights
+    if not lora_weights:
+        raise ValueError(f'can not find lora weight, the value is: {lora_weights}')
+
+    load_8bit = args.load_8bit
+    if args.model == 'LLaMA-7B':
+        tokenizer = LlamaTokenizer.from_pretrained(base_model)
+    else:
+        tokenizer = AutoTokenizer.from_pretrained(base_model)
+    if device == "cuda":
+        model = AutoModelForCausalLM.from_pretrained(
+            base_model,
+            load_in_8bit=load_8bit,
+            torch_dtype=torch.float16,
+            device_map="auto",
+            trust_remote_code=True,
+        ) # fix zwq
+        model = PeftModel.from_pretrained(
+            model,
+            lora_weights,
+            torch_dtype=torch.float16,
+            device_map={"":0}
+        )
+    elif device == "mps":
+        model = AutoModelForCausalLM.from_pretrained(
+            base_model,
+            device_map={"": device},
+            torch_dtype=torch.float16,
+        )
+        model = PeftModel.from_pretrained(
+            model,
+            lora_weights,
+            device_map={"": device},
+            torch_dtype=torch.float16,
+        )
+    else:
+        model = AutoModelForCausalLM.from_pretrained(
+            base_model, device_map={"": device}, low_cpu_mem_usage=True
+        )
+        model = PeftModel.from_pretrained(
+            model,
+            lora_weights,
+            device_map={"": device},
+        )
+
+        # unwind broken decapoda-research config
+        model.config.pad_token_id = tokenizer.pad_token_id = 0  # unk
+        model.config.bos_token_id = 1
+        model.config.eos_token_id = 2
+
+        if not load_8bit:
+            model.half()  # seems to fix bugs for some users.
+
+        model.eval()
+        if torch.__version__ >= "2" and sys.platform != "win32":
+            model = torch.compile(model)
+
+    return tokenizer, model
+
+
+def load_instruction(args) -> str:
+    instruction = ''
+    if not instruction:
+        raise ValueError('instruct not initialized')
+    return instruction
+
+
+def extract_answer_number(args, sentence: str) -> float:
+    dataset = args.dataset.lower()
+    if dataset in ["multiarith", "addsub", "singleeq", "gsm8k", "svamp"]:
+        sentence = sentence.replace(',', '')
+        pred = [s for s in re.findall(r'-?\d+\.?\d*', sentence)]
+        if not pred:
+            return float('inf')
+        pred_answer = float(pred[-1])
+    else:
+        raise NotImplementedError(' not support dataset: {}'.format(dataset))
+    if isinstance(pred_answer, str):
+        try:
+            pred_answer = float(pred_answer)
+        except ValueError as e:
+            pred_answer = float('inf')
+    return pred_answer
+
+
+def extract_answer_letter(args, sentence: str) -> str:
+    sentence_ = sentence.strip()
+    pred_answers = re.findall(r'A|B|C|D|E', sentence_)
+    if pred_answers:
+        if not pred_answers:
+            return ''
+        return pred_answers[0]
+    else:
+        return ''
+
+
+if __name__ == "__main__":
+    fire.Fire(main)

SVFT-main/LLM-Adapters/export_hf_checkpoint.py

@@ -0,0 +1,57 @@
+import os
+
+import torch
+import transformers
+from peft import PeftModel
+from transformers import LlamaForCausalLM, LlamaTokenizer  # noqa: F402
+
+BASE_MODEL = os.environ.get("BASE_MODEL", None)
+assert (
+    BASE_MODEL
+), "Please specify a value for BASE_MODEL environment variable, e.g. `export BASE_MODEL=decapoda-research/llama-7b-hf`"  # noqa: E501
+
+tokenizer = LlamaTokenizer.from_pretrained(BASE_MODEL)
+
+base_model = LlamaForCausalLM.from_pretrained(
+    BASE_MODEL,
+    load_in_8bit=False,
+    torch_dtype=torch.float16,
+    device_map={"": "cpu"},
+)
+
+first_weight = base_model.model.layers[0].self_attn.q_proj.weight
+first_weight_old = first_weight.clone()
+
+lora_model = PeftModel.from_pretrained(
+    base_model,
+    "tloen/alpaca-lora-7b",
+    device_map={"": "cpu"},
+    torch_dtype=torch.float16,
+)
+
+lora_weight = lora_model.base_model.model.model.layers[
+    0
+].self_attn.q_proj.weight
+
+assert torch.allclose(first_weight_old, first_weight)
+
+# merge weights
+for layer in lora_model.base_model.model.model.layers:
+    layer.self_attn.q_proj.merge_weights = True
+    layer.self_attn.v_proj.merge_weights = True
+
+lora_model.train(False)
+
+# did we do anything?
+assert not torch.allclose(first_weight_old, first_weight)
+
+lora_model_sd = lora_model.state_dict()
+deloreanized_sd = {
+    k.replace("base_model.model.", ""): v
+    for k, v in lora_model_sd.items()
+    if "lora" not in k
+}
+
+LlamaForCausalLM.save_pretrained(
+    base_model, "./hf_ckpt", state_dict=deloreanized_sd, max_shard_size="400MB"
+)

SVFT-main/LLM-Adapters/export_state_dict_checkpoint.py

@@ -0,0 +1,125 @@
+import json
+import os
+
+import torch
+import transformers
+from peft import PeftModel
+from transformers import LlamaForCausalLM, LlamaTokenizer  # noqa: E402
+
+BASE_MODEL = os.environ.get("BASE_MODEL", None)
+assert (
+    BASE_MODEL
+), "Please specify a value for BASE_MODEL environment variable, e.g. `export BASE_MODEL=decapoda-research/llama-7b-hf`"  # noqa: E501
+
+tokenizer = LlamaTokenizer.from_pretrained(BASE_MODEL)
+
+base_model = LlamaForCausalLM.from_pretrained(
+    BASE_MODEL,
+    load_in_8bit=False,
+    torch_dtype=torch.float16,
+    device_map={"": "cpu"},
+)
+
+lora_model = PeftModel.from_pretrained(
+    base_model,
+    "tloen/alpaca-lora-7b",
+    device_map={"": "cpu"},
+    torch_dtype=torch.float16,
+)
+
+# merge weights
+for layer in lora_model.base_model.model.model.layers:
+    layer.self_attn.q_proj.merge_weights = True
+    layer.self_attn.v_proj.merge_weights = True
+
+lora_model.train(False)
+
+lora_model_sd = lora_model.state_dict()
+
+params = {
+    "dim": 4096,
+    "multiple_of": 256,
+    "n_heads": 32,
+    "n_layers": 32,
+    "norm_eps": 1e-06,
+    "vocab_size": -1,
+}
+n_layers = params["n_layers"]
+n_heads = params["n_heads"]
+dim = params["dim"]
+dims_per_head = dim // n_heads
+base = 10000.0
+inv_freq = 1.0 / (
+    base ** (torch.arange(0, dims_per_head, 2).float() / dims_per_head)
+)
+
+
+def permute(w):
+    return (
+        w.view(n_heads, dim // n_heads // 2, 2, dim)
+        .transpose(1, 2)
+        .reshape(dim, dim)
+    )
+
+
+def unpermute(w):
+    return (
+        w.view(n_heads, 2, dim // n_heads // 2, dim)
+        .transpose(1, 2)
+        .reshape(dim, dim)
+    )
+
+
+def translate_state_dict_key(k):  # noqa: C901
+    k = k.replace("base_model.model.", "")
+    if k == "model.embed_tokens.weight":
+        return "tok_embeddings.weight"
+    elif k == "model.norm.weight":
+        return "norm.weight"
+    elif k == "lm_head.weight":
+        return "output.weight"
+    elif k.startswith("model.layers."):
+        layer = k.split(".")[2]
+        if k.endswith(".self_attn.q_proj.weight"):
+            return f"layers.{layer}.attention.wq.weight"
+        elif k.endswith(".self_attn.k_proj.weight"):
+            return f"layers.{layer}.attention.wk.weight"
+        elif k.endswith(".self_attn.v_proj.weight"):
+            return f"layers.{layer}.attention.wv.weight"
+        elif k.endswith(".self_attn.o_proj.weight"):
+            return f"layers.{layer}.attention.wo.weight"
+        elif k.endswith(".mlp.gate_proj.weight"):
+            return f"layers.{layer}.feed_forward.w1.weight"
+        elif k.endswith(".mlp.down_proj.weight"):
+            return f"layers.{layer}.feed_forward.w2.weight"
+        elif k.endswith(".mlp.up_proj.weight"):
+            return f"layers.{layer}.feed_forward.w3.weight"
+        elif k.endswith(".input_layernorm.weight"):
+            return f"layers.{layer}.attention_norm.weight"
+        elif k.endswith(".post_attention_layernorm.weight"):
+            return f"layers.{layer}.ffn_norm.weight"
+        elif k.endswith("rotary_emb.inv_freq") or "lora" in k:
+            return None
+        else:
+            print(layer, k)
+            raise NotImplementedError
+    else:
+        print(k)
+        raise NotImplementedError
+
+
+new_state_dict = {}
+for k, v in lora_model_sd.items():
+    new_k = translate_state_dict_key(k)
+    if new_k is not None:
+        if "wq" in new_k or "wk" in new_k:
+            new_state_dict[new_k] = unpermute(v)
+        else:
+            new_state_dict[new_k] = v
+
+os.makedirs("./ckpt", exist_ok=True)
+
+torch.save(new_state_dict, "./ckpt/consolidated.00.pth")
+
+with open("./ckpt/params.json", "w") as f:
+    json.dump(params, f)

SVFT-main/LLM-Adapters/finetune.py

@@ -0,0 +1,438 @@
+import os
+import sys
+from typing import List
+
+import fire
+import torch
+import argparse
+import transformers
+from datasets import load_dataset
+from typing import List, Optional, Union
+
+from tqdm import tqdm
+import sys
+from functools import partial, reduce
+sys.path.append("../")
+from svft.svft_layers import LinearWithSVFT, create_and_replace_modules, get_target_modules_list, replace_svft_with_fused_linear
+
+"""
+Unused imports:
+import torch.nn as nn
+import bitsandbytes as bnb
+"""
+sys.path.append(os.path.join(os.getcwd(), "peft/src/"))
+
+from peft import (  # noqa: E402
+    LoraConfig, BOFTConfig, VeraConfig, 
+    PrefixTuningConfig,
+    get_peft_model,
+    get_peft_model_state_dict,
+    set_peft_model_state_dict,
+)
+from transformers import AutoModelForCausalLM, AutoTokenizer, LlamaTokenizer, AutoModel  # noqa: F402
+
+
+def train(
+        # model/data params
+        base_model: str = "",  # the only required argument
+        data_path: str = "yahma/alpaca-cleaned",
+        output_dir: str = "./lora-alpaca",
+        adapter_name: str = "lora",
+        load_8bit : bool = False,
+        # training hyperparams
+        batch_size: int = 128,
+        micro_batch_size: int = 4,
+        num_epochs: int = 3,
+        learning_rate: float = 3e-4,
+        cutoff_len: int = 256,
+        val_set_size: int = 2000,
+        use_gradient_checkpointing: bool = False,
+        eval_step: int = 200,
+        save_step: int = 200,
+        # lora hyperparams
+        lora_r: int = None,
+        lora_alpha: int = 16,
+        lora_dropout: float = 0.05,
+        lora_target_modules: List[str] = None,
+        # bottleneck adapter hyperparams
+        bottleneck_size: int = 256,
+        non_linearity: str = "tanh",
+        adapter_dropout: float = 0.0,
+        use_parallel_adapter: bool = False,
+        use_adapterp: bool = False,
+        target_modules: List[str] = None,
+        scaling: Union[float, str] = 1.0,
+        # prefix tuning hyperparams
+        num_virtual_tokens: int = 30,
+        # llm hyperparams
+        train_on_inputs: bool = True,  # if False, masks out inputs in loss
+        group_by_length: bool = False,  # faster, but produces an odd training loss curve
+        # wandb params
+        wandb_project: str = "",
+        wandb_run_name: str = "",
+        wandb_watch: str = "",  # options: false | gradients | all
+        wandb_log_model: str = "",  # options: false | true
+        resume_from_checkpoint: str = None,  # either training checkpoint or final adapter
+        off_diag: int = 0,
+        pattern: str = "banded",
+        fill_orthonormal: bool = False,
+):
+    print(
+        f"Finetuning model with params:\n"
+        f"base_model: {base_model}\n"
+        f"data_path: {data_path}\n"
+        f"output_dir: {output_dir}\n"
+        f"batch_size: {batch_size}\n"
+        f"micro_batch_size: {micro_batch_size}\n"
+        f"num_epochs: {num_epochs}\n"
+        f"learning_rate: {learning_rate}\n"
+        f"cutoff_len: {cutoff_len}\n"
+        f"val_set_size: {val_set_size}\n"
+        f"use_gradient_checkpointing: {use_gradient_checkpointing}\n"
+        f"lora_r: {lora_r}\n"
+        f"lora_alpha: {lora_alpha}\n"
+        f"lora_dropout: {lora_dropout}\n"
+        f"lora_target_modules: {lora_target_modules}\n"
+        f"bottleneck_size: {bottleneck_size}\n"
+        f"non_linearity: {non_linearity}\n"
+        f"adapter_dropout: {adapter_dropout}\n"
+        f"use_parallel_adapter: {use_parallel_adapter}\n"
+        f"use_adapterp: {use_adapterp}\n"
+        f"train_on_inputs: {train_on_inputs}\n"
+        f"scaling: {scaling}\n"
+        f"adapter_name: {adapter_name}\n"
+        f"target_modules: {target_modules}\n"
+        f"group_by_length: {group_by_length}\n"
+        f"wandb_project: {wandb_project}\n"
+        f"wandb_run_name: {wandb_run_name}\n"
+        f"wandb_watch: {wandb_watch}\n"
+        f"wandb_log_model: {wandb_log_model}\n"
+        f"resume_from_checkpoint: {resume_from_checkpoint}\n"
+    )
+
+    print(base_model)
+
+    # assert (
+    #     base_model
+    # ), "Please specify a --base_model, e.g. --base_model='decapoda-research/llama-7b-hf'"
+    gradient_accumulation_steps = batch_size // micro_batch_size
+
+    device_map = "auto"
+    world_size = int(os.environ.get("WORLD_SIZE", 1))
+    ddp = world_size != 1
+    if ddp:
+        device_map = {"": int(os.environ.get("LOCAL_RANK") or 0)}
+        gradient_accumulation_steps = gradient_accumulation_steps // world_size
+
+    # Check if parameter passed or if set within environ
+    use_wandb = len(wandb_project) > 0 or (
+            "WANDB_PROJECT" in os.environ and len(os.environ["WANDB_PROJECT"]) > 0
+    )
+    # Only overwrite environ if wandb param passed
+    if len(wandb_project) > 0:
+        os.environ["WANDB_PROJECT"] = "CommonsenseReasoning"
+    if len(wandb_watch) > 0:
+        os.environ["WANDB_WATCH"] = "all"
+    if len(wandb_log_model) > 0:
+        os.environ["WANDB_LOG_MODEL"] = False
+
+    if load_8bit:
+        model = AutoModelForCausalLM.from_pretrained(
+            base_model,
+            load_in_8bit=load_8bit,
+            torch_dtype=torch.float16,
+            device_map=device_map,
+            trust_remote_code=True,
+        )
+    else:
+        model = AutoModelForCausalLM.from_pretrained(
+            base_model,
+            load_in_8bit=False,
+            torch_dtype=torch.float32,
+            device_map={"": int(os.environ.get("LOCAL_RANK") or 0)},
+            trust_remote_code=True,
+            #revision="step143000",
+        )
+
+    if model.config.model_type == "llama":
+        # Due to the name of transformers' LlamaTokenizer, we have to do this
+        tokenizer = LlamaTokenizer.from_pretrained(base_model)
+    else:
+        tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)
+
+    tokenizer.pad_token_id = (
+        0  # unk. we want this to be different from the eos token
+    )
+    tokenizer.padding_side = "left"  # Allow batched inference
+
+    def tokenize(prompt, add_eos_token=True):
+        # there's probably a way to do this with the tokenizer settings
+        # but again, gotta move fast
+        result = tokenizer(
+            prompt,
+            truncation=True,
+            max_length=cutoff_len,
+            padding=False,
+            return_tensors=None,
+        )
+        if (
+                result["input_ids"][-1] != tokenizer.eos_token_id
+                and len(result["input_ids"]) < cutoff_len
+                and add_eos_token
+        ):
+            result["input_ids"].append(tokenizer.eos_token_id)
+            if "chatglm" not in base_model:
+                result["attention_mask"].append(1)
+
+        result["labels"] = result["input_ids"].copy()
+
+        if "chatglm" in base_model:
+            return {"input_ids": result["input_ids"], "labels": result["labels"]}
+        else:
+            return result
+
+    def generate_and_tokenize_prompt(data_point):
+        full_prompt = generate_prompt(data_point)
+        tokenized_full_prompt = tokenize(full_prompt)
+        if not train_on_inputs:
+            user_prompt = generate_prompt({**data_point, "output": ""})
+            tokenized_user_prompt = tokenize(user_prompt, add_eos_token=False)
+            user_prompt_len = len(tokenized_user_prompt["input_ids"])
+
+            tokenized_full_prompt["labels"] = [
+                                                  -100
+                                              ] * user_prompt_len + tokenized_full_prompt["labels"][
+                                                                    user_prompt_len:
+                                                                    ]  # could be sped up, probably
+        return tokenized_full_prompt
+
+    if adapter_name == "lora":
+        config = LoraConfig(
+            r=lora_r,
+            lora_alpha=lora_alpha,
+            target_modules=lora_target_modules,
+            lora_dropout=lora_dropout,
+            bias="none",
+            task_type="CAUSAL_LM",
+        )
+
+    elif adapter_name == "dora":
+        config = LoraConfig(
+            use_dora=True,
+            r=lora_r,
+            lora_alpha=lora_alpha,
+            target_modules=lora_target_modules,
+            lora_dropout=lora_dropout,
+            bias="none",
+            task_type="CAUSAL_LM",
+        ) 
+    
+    elif adapter_name == "boft":
+        config = BOFTConfig(
+            boft_block_size=8,
+            boft_n_butterfly_factor=2,
+            target_modules=lora_target_modules,
+            boft_dropout=0.05,
+            bias="boft_only",
+        )
+
+    elif adapter_name == "boft_r1":
+        config = BOFTConfig(
+            boft_block_size=1,
+            boft_n_butterfly_factor=1,
+            target_modules=lora_target_modules,
+            boft_dropout=0.05,
+            bias="boft_only",
+        )
+
+    elif adapter_name == "vera":
+        config = VeraConfig(r=lora_r, target_modules=lora_target_modules)
+
+    if adapter_name == 'svft':
+        # for SVFT turn off gradient requirement for all layers
+        # PEFT library handles this internally
+        for param in model.parameters():
+            param.requires_grad = False
+
+        print(f"Target Modules: {lora_target_modules}")
+        assign_svft_layer = partial(LinearWithSVFT, 
+                                    off_diag=off_diag, 
+                                    pattern=pattern, 
+                                    rank=lora_r, 
+                                    fill_orthonormal=fill_orthonormal)
+        
+        create_and_replace_modules(model, get_target_modules_list(model, lora_target_modules), assign_svft_layer)
+
+    elif adapter_name == "full_ft":
+        pass
+    else:
+        # for baseline peft models    
+        model = get_peft_model(model, config)
+
+    if adapter_name == "prefix-tuning":
+        model.to('cuda')
+
+    if data_path.endswith(".json"):  # todo: support jsonl
+        data = load_dataset("json", data_files=data_path)
+    else:
+        data = load_dataset(data_path)
+
+    print(f"Trainable Parameters: {sum(p.numel() for p in model.parameters() if p.requires_grad)}")
+    print(f"Output Dir: {output_dir}")
+
+    if val_set_size > 0:
+        train_val = data["train"].train_test_split(
+            test_size=val_set_size, shuffle=True, seed=42
+        )
+        train_data = (
+            train_val["train"].shuffle().map(generate_and_tokenize_prompt)
+        )
+        val_data = (
+            train_val["test"].shuffle().map(generate_and_tokenize_prompt)
+        )
+    else:
+        train_data = data["train"].shuffle().map(generate_and_tokenize_prompt)
+        val_data = None
+
+    if not ddp and torch.cuda.device_count() > 1:
+        # keeps Trainer from trying its own DataParallelism when more than 1 gpu is available
+        model.is_parallelizable = True
+        model.model_parallel = True
+
+    trainer = transformers.Trainer(
+        model=model,
+        train_dataset=train_data,
+        eval_dataset=val_data,
+        args=transformers.TrainingArguments(
+            per_device_train_batch_size=micro_batch_size,
+            gradient_accumulation_steps=gradient_accumulation_steps,
+            warmup_steps=100,
+            num_train_epochs=num_epochs,
+            learning_rate=learning_rate,
+            bf16=True,
+            logging_steps=10,
+            optim="adamw_torch",
+            evaluation_strategy="steps" if val_set_size > 0 else "no",
+            save_strategy="steps",
+            eval_steps=eval_step if val_set_size > 0 else None,
+            save_steps=save_step,
+            output_dir=output_dir,
+            save_total_limit=3,
+            load_best_model_at_end=False if val_set_size > 0 else False,
+            ddp_find_unused_parameters=False if ddp else None,
+            group_by_length=group_by_length,
+            report_to="wandb" if use_wandb else None,
+            run_name=wandb_run_name if use_wandb else None,
+            #deepspeed="deepspeed.json"
+        ),
+        data_collator=transformers.DataCollatorForSeq2Seq(
+            tokenizer, pad_to_multiple_of=8, return_tensors="pt", padding=True
+        ),
+    )
+    model.config.use_cache = False
+
+    if adapter_name not in ['boft', 'svft']:
+        model = model.bfloat16()
+
+    trainer.train(resume_from_checkpoint=resume_from_checkpoint)
+
+    model.generation_config.temperature = 1.0
+    model.generation_config.top_p = 1.0
+    
+    if adapter_name == 'svft':
+        replace_svft_with_fused_linear(model, get_target_modules_list(model, lora_target_modules))
+    elif adapter_name=="full_ft":
+        pass
+    else:
+        model = model.merge_and_unload()
+
+    for param in model.parameters():
+        param.data = param.data.contiguous()
+    model.save_pretrained(output_dir)
+    tokenizer.save_pretrained(output_dir)
+
+    print(
+        "\n If there's a warning about missing keys above, please disregard :)"
+    )
+
+
+def generate_prompt(data_point):
+    # sorry about the formatting disaster gotta move fast
+    if data_point["input"]:
+        return f"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. 
+
+                ### Instruction:
+                {data_point["instruction"]}
+                
+                ### Input:
+                {data_point["input"]}
+                
+                ### Response:
+                {data_point["output"]}""" # noqa: E501
+    else:
+        return f"""Below is an instruction that describes a task. Write a response that appropriately completes the request.  
+
+                ### Instruction:
+                {data_point["instruction"]}
+                
+                ### Response:
+                {data_point["output"]}""" # noqa: E501
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(description='Train a model')
+    
+    # model/data params
+    parser.add_argument('--base_model', type=str, required=True, help='Base model')
+    parser.add_argument('--data_path', type=str, default='yahma/alpaca-cleaned', help='Data path')
+    parser.add_argument('--output_dir', type=str, default='./lora-alpaca', help='Output directory')
+    parser.add_argument('--adapter_name', type=str, default='lora', help='Adapter name')
+    parser.add_argument('--load_8bit', action='store_true', help='Load 8-bit')
+    
+    # training hyperparams
+    parser.add_argument('--batch_size', type=int, default=128, help='Batch size')
+    parser.add_argument('--micro_batch_size', type=int, default=4, help='Micro batch size')
+    parser.add_argument('--num_epochs', type=int, default=3, help='Number of epochs')
+    parser.add_argument('--learning_rate', type=float, default=3e-4, help='Learning rate')
+    parser.add_argument('--cutoff_len', type=int, default=256, help='Cutoff length')
+    parser.add_argument('--val_set_size', type=int, default=2000, help='Validation set size')
+    parser.add_argument('--use_gradient_checkpointing', action='store_true', help='Use gradient checkpointing')
+    parser.add_argument('--eval_step', type=int, default=200, help='Evaluation step')
+    parser.add_argument('--save_step', type=int, default=200, help='Save step')
+    
+    # lora hyperparams
+    parser.add_argument('--lora_r', type=int, default=8, help='Lora r')
+    parser.add_argument('--lora_alpha', type=int, default=16, help='Lora alpha')
+    parser.add_argument('--lora_dropout', type=float, default=0.05, help='Lora dropout')
+    parser.add_argument('--lora_target_modules', nargs='+', help='Lora target modules')
+    
+    # bottleneck adapter hyperparams
+    parser.add_argument('--bottleneck_size', type=int, default=256, help='Bottleneck size')
+    parser.add_argument('--non_linearity', type=str, default='tanh', help='Non-linearity')
+    parser.add_argument('--adapter_dropout', type=float, default=0.0, help='Adapter dropout')
+    parser.add_argument('--use_parallel_adapter', action='store_true', help='Use parallel adapter')
+    parser.add_argument('--use_adapterp', action='store_true', help='Use adapterp')
+    parser.add_argument('--target_modules', nargs='+', help='Target modules')
+    parser.add_argument('--scaling', type=Union[float, str], default=1.0, help='Scaling')
+    
+    # prefix tuning hyperparams
+    parser.add_argument('--num_virtual_tokens', type=int, default=30, help='Number of virtual tokens')
+    
+    # llm hyperparams
+    parser.add_argument('--train_on_inputs', action='store_true', help='Train on inputs')
+    parser.add_argument('--group_by_length', action='store_true', help='Group by length')
+    
+    # wandb params
+    parser.add_argument('--wandb_project', type=str, default='', help='Wandb project')
+    parser.add_argument('--wandb_run_name', type=str, default='', help='Wandb run name')
+    parser.add_argument('--wandb_watch', type=str, default='', help='Wandb watch')
+    parser.add_argument('--wandb_log_model', type=str, default='', help='Wandb log model')
+    parser.add_argument('--resume_from_checkpoint', type=str, help='Resume from checkpoint')
+
+    return parser.parse_args()
+
+if __name__ == "__main__":
+    fire.Fire(train)
+
+    # args = parse_args()
+    # train(**vars(args))

SVFT-main/LLM-Adapters/generate.py

@@ -0,0 +1,191 @@
+import os
+import sys
+
+import fire
+import gradio as gr
+import torch
+import transformers
+from peft import PeftModel
+from transformers import GenerationConfig, LlamaForCausalLM, LlamaTokenizer
+
+if torch.cuda.is_available():
+    device = "cuda"
+else:
+    device = "cpu"
+
+try:
+    if torch.backends.mps.is_available():
+        device = "mps"
+except:  # noqa: E722
+    pass
+
+
+def main(
+    load_8bit: bool = False,
+    base_model: str = "",
+    lora_weights: str = "tloen/alpaca-lora-7b",
+    share_gradio: bool = False,
+):
+    assert (
+        base_model
+    ), "Please specify a --base_model, e.g. --base_model='decapoda-research/llama-7b-hf'"
+
+    tokenizer = LlamaTokenizer.from_pretrained(base_model)
+    if device == "cuda":
+        model = LlamaForCausalLM.from_pretrained(
+            base_model,
+            load_in_8bit=load_8bit,
+            torch_dtype=torch.float16,
+            device_map="auto",
+            trust_remote_code=True,
+        )
+        model = PeftModel.from_pretrained(
+            model,
+            lora_weights,
+            torch_dtype=torch.float16,
+        )
+    elif device == "mps":
+        model = LlamaForCausalLM.from_pretrained(
+            base_model,
+            device_map={"": device},
+            torch_dtype=torch.float16,
+        )
+        model = PeftModel.from_pretrained(
+            model,
+            lora_weights,
+            device_map={"": device},
+            torch_dtype=torch.float16,
+        )
+    else:
+        model = LlamaForCausalLM.from_pretrained(
+            base_model, device_map={"": device}, low_cpu_mem_usage=True
+        )
+        model = PeftModel.from_pretrained(
+            model,
+            lora_weights,
+            device_map={"": device},
+        )
+
+    # unwind broken decapoda-research config
+    model.config.pad_token_id = tokenizer.pad_token_id = 0  # unk
+    model.config.bos_token_id = 1
+    model.config.eos_token_id = 2
+
+    if not load_8bit:
+        model.half()  # seems to fix bugs for some users.
+
+    model.eval()
+    if torch.__version__ >= "2" and sys.platform != "win32":
+        model = torch.compile(model)
+
+    def evaluate(
+        instruction,
+        input=None,
+        temperature=0.1,
+        top_p=0.75,
+        top_k=40,
+        num_beams=4,
+        max_new_tokens=128,
+        **kwargs,
+    ):
+        prompt = generate_prompt(instruction, input)
+        inputs = tokenizer(prompt, return_tensors="pt")
+        input_ids = inputs["input_ids"].to(device)
+        generation_config = GenerationConfig(
+            temperature=temperature,
+            top_p=top_p,
+            top_k=top_k,
+            num_beams=num_beams,
+            **kwargs,
+        )
+        with torch.no_grad():
+            generation_output = model.generate(
+                input_ids=input_ids,
+                generation_config=generation_config,
+                return_dict_in_generate=True,
+                output_scores=True,
+                max_new_tokens=max_new_tokens,
+            )
+        s = generation_output.sequences[0]
+        output = tokenizer.decode(s)
+        return output.split("### Response:")[1].strip()
+
+    gr.Interface(
+        fn=evaluate,
+        inputs=[
+            gr.components.Textbox(
+                lines=2,
+                label="Instruction",
+                placeholder="Tell me about alpacas.",
+            ),
+            gr.components.Textbox(lines=2, label="Input", placeholder="none"),
+            gr.components.Slider(
+                minimum=0, maximum=1, value=0.1, label="Temperature"
+            ),
+            gr.components.Slider(
+                minimum=0, maximum=1, value=0.75, label="Top p"
+            ),
+            gr.components.Slider(
+                minimum=0, maximum=100, step=1, value=40, label="Top k"
+            ),
+            gr.components.Slider(
+                minimum=1, maximum=4, step=1, value=4, label="Beams"
+            ),
+            gr.components.Slider(
+                minimum=1, maximum=2000, step=1, value=128, label="Max tokens"
+            ),
+        ],
+        outputs=[
+            gr.inputs.Textbox(
+                lines=5,
+                label="Output",
+            )
+        ],
+        title="LLM-Adapters",
+        description="This is a 7B-parameter LLaMA model finetuned to follow instructions. It is trained on the [Stanford Alpaca](https://github.com/tatsu-lab/stanford_alpaca) dataset and makes use of the Huggingface LLaMA implementation.",  # noqa: E501
+    ).launch(share=share_gradio)
+    # Old testing code follows.
+
+    """
+    # testing code for readme
+    for instruction in [
+        "Tell me about alpacas.",
+        "Tell me about the president of Mexico in 2019.",
+        "Tell me about the king of France in 2019.",
+        "List all Canadian provinces in alphabetical order.",
+        "Write a Python program that prints the first 10 Fibonacci numbers.",
+        "Write a program that prints the numbers from 1 to 100. But for multiples of three print 'Fizz' instead of the number and for the multiples of five print 'Buzz'. For numbers which are multiples of both three and five print 'FizzBuzz'.",  # noqa: E501
+        "Tell me five words that rhyme with 'shock'.",
+        "Translate the sentence 'I have no mouth but I must scream' into Spanish.",
+        "Count up from 1 to 500.",
+    ]:
+        print("Instruction:", instruction)
+        print("Response:", evaluate(instruction))
+        print()
+    """
+
+
+def generate_prompt(instruction, input=None):
+    if input:
+        return f"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. 
+
+### Instruction:
+{instruction}
+
+### Input:
+{input}
+
+### Response:
+""" # noqa: E501
+    else:
+        return f"""Below is an instruction that describes a task. Write a response that appropriately completes the request.  
+
+### Instruction:
+{instruction}
+
+### Response:
+"""  # noqa: E501
+
+
+if __name__ == "__main__":
+    fire.Fire(main)

SVFT-main/LLM-Adapters/lengths.ipynb

@@ -0,0 +1,204 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/home/eric/miniconda3/envs/dl3/lib/python3.10/site-packages/tqdm/auto.py:22: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
+      "  from .autonotebook import tqdm as notebook_tqdm\n",
+      "Found cached dataset json (/home/eric/.cache/huggingface/datasets/json/default-789f51900889f651/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51)\n",
+      "100%|██████████| 1/1 [00:00<00:00, 784.28it/s]\n",
+      "Loading cached processed dataset at /home/eric/.cache/huggingface/datasets/json/default-789f51900889f651/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-f691ee34ec2034cb.arrow\n"
+     ]
+    }
+   ],
+   "source": [
+    "from datasets import load_dataset\n",
+    "from transformers import LlamaTokenizer\n",
+    "\n",
+    "\n",
+    "tokenizer = LlamaTokenizer.from_pretrained(\n",
+    "    \"decapoda-research/llama-7b-hf\", add_eos_token=True\n",
+    ")\n",
+    "tokenizer.pad_token = tokenizer.eos_token\n",
+    "tokenizer.pad_token_id = tokenizer.eos_token_id\n",
+    "\n",
+    "data = load_dataset(\"json\", data_files=\"alpaca_data.json\")\n",
+    "\n",
+    "\n",
+    "def generate_prompt(data_point):\n",
+    "    # sorry about the formatting disaster gotta move fast\n",
+    "    if data_point[\"input\"]:\n",
+    "        return f\"\"\"Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n",
+    "\n",
+    "### Instruction:\n",
+    "{data_point[\"instruction\"]}\n",
+    "\n",
+    "### Input:\n",
+    "{data_point[\"input\"]}\n",
+    "\n",
+    "### Response:\n",
+    "{data_point[\"output\"]}\"\"\"\n",
+    "    else:\n",
+    "        return f\"\"\"Below is an instruction that describes a task. Write a response that appropriately completes the request.\n",
+    "\n",
+    "### Instruction:\n",
+    "{data_point[\"instruction\"]}\n",
+    "\n",
+    "### Response:\n",
+    "{data_point[\"output\"]}\"\"\"\n",
+    "\n",
+    "\n",
+    "data = data.map(\n",
+    "    lambda data_point: {\"prompt\": tokenizer(generate_prompt(data_point))}\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "<matplotlib.lines.Line2D at 0x7f6f1af20af0>"
+      ]
+     },
+     "execution_count": 2,
+     "metadata": {},
+     "output_type": "execute_result"
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjAAAAGzCAYAAAAxPS2EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA/q0lEQVR4nO3de1hVZf7//xeobBDcG0+AJCqTjkoeSizd09Ekyeio1mhmaJpfDSu0MbUaM5vS0UrtoFbOiJ/ppHaVleSBxMOUpEZRamk6YTgZWBls8QAK9++PfqxxCyoQCAuej+va18W+13uvfa8bkJf3WvfaPsYYIwAAABvxrekOAAAAVBQBBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBvXWtGnT5OPjc17e65prrtE111xjPd+wYYN8fHz09ttvn5f3Hz58uNq1a3de3quy8vPzNWrUKIWFhcnHx0eJiYk13aVareRnaMOGDTXdlXPy8fHRuHHjarobqGMIMKgTkpKS5OPjYz38/f0VHh6u2NhYPf/88zp8+HCVvM+BAwc0bdo0ZWRkVMn+qlJt7lt5PP3000pKStLYsWP1r3/9S8OGDavpLp1XR48e1bRp02wRSMqyefNmTZs2Tbm5uTXdFdQTDWu6A0BVmj59uiIjI3XixAllZ2drw4YNSkxM1HPPPaf3339f3bp1s2ofe+wxTZ48uUL7P3DggJ544gm1a9dOF198cblft3bt2gq9T2WcrW+vvvqqiouLq70Pv0dqaqp69+6txx9/vKa7UiOOHj2qJ554QpK8ZuvsYvPmzXriiSc0fPhwBQcH13R3UA8QYFCn9O/fXz179rSeT5kyRampqbrxxht1880365tvvlFAQIAkqWHDhmrYsHp/BY4eParGjRvLz8+vWt/nXBo1alSj718eBw8eVFRUVLW/z5EjRxQYGFjt7wOgenEKCXXetddeq7/+9a/6/vvv9dprr1ntZV0Dk5KSoiuuuELBwcEKCgpSx44d9cgjj0j67ZqDSy+9VJI0YsQI63RVUlKSpN/+19ylSxelp6frqquuUuPGja3Xnn4NTImioiI98sgjCgsLU2BgoG6++Wbt37/fq6Zdu3YaPnx4qdeeus9z9a2sa2COHDmihx56SBEREXI4HOrYsaOeeeYZnf4B9SXXL6xYsUJdunSRw+HQRRddpNWrV5c94Kc5ePCgRo4cqdDQUPn7+6t79+5asmSJtb3kWo7MzEwlJydbfd+3b98Z91nSp9dff10dO3aUv7+/oqOjtWnTJq+6ku/x119/rTvvvFNNmzbVFVdcIUk6efKknnzySV144YVyOBxq166dHnnkERUUFHjto127drrxxhu1YcMG9ezZUwEBAeratat1quedd95R165drT588cUXXq8fPny4goKC9N133yk2NlaBgYEKDw/X9OnTrbHet2+fWrZsKUl64oknrDGYNm1aucb4VFu2bNH1118vl8ulxo0b6+qrr9Ynn3xS5rjs3bvXmjFxuVwaMWKEjh496lV77NgxPfDAA2rRooWaNGmim2++WT/88INX/6ZNm6aJEydKkiIjI8/4PTzXz9Dhw4eVmJiodu3ayeFwKCQkRNddd50+//zzCo8D6j5mYFAvDBs2TI888ojWrl2re++9t8yanTt36sYbb1S3bt00ffp0ORwO7d271/rHv3Pnzpo+fbqmTp2q0aNH68orr5Qk/elPf7L28csvv6h///4aPHiw7rrrLoWGhp61X0899ZR8fHw0adIkHTx4UHPnzlVMTIwyMjKsmaLyKE/fTmWM0c0336z169dr5MiRuvjii7VmzRpNnDhRP/zwg+bMmeNV//HHH+udd97RfffdpyZNmuj555/XwIEDlZWVpebNm5+xX8eOHdM111yjvXv3aty4cYqMjNTy5cs1fPhw5ebm6sEHH1Tnzp31r3/9S+PHj1fr1q310EMPSZL1B/1MNm7cqKVLl+qBBx6Qw+HQ/Pnzdf3112vr1q3q0qWLV+3tt9+uDh066Omnn7ZCw6hRo7RkyRINGjRIDz30kLZs2aIZM2bom2++0bvvvuv1+r179+rOO+/U//t//0933XWXnnnmGd10001auHChHnnkEd13332SpBkzZuiOO+7Q7t275ev7v/8fFhUV6frrr1fv3r01a9YsrV69Wo8//rhOnjyp6dOnq2XLllqwYIHGjh2r2267TQMGDJAkr1Oe5ZGamqr+/fsrOjpajz/+uHx9fbV48WJde+21+ve//63LLrvMq/6OO+5QZGSkZsyYoc8//1yLFi1SSEiI/v73v1s1w4cP17JlyzRs2DD17t1bGzduVFxcnNd+BgwYoG+//VZvvvmm5syZoxYtWkjy/h6W52dozJgxevvttzVu3DhFRUXpl19+0ccff6xvvvlGPXr0qNBYoB4wQB2wePFiI8ls27btjDUul8tccskl1vPHH3/cnPorMGfOHCPJ/PTTT2fcx7Zt24wks3jx4lLbrr76aiPJLFy4sMxtV199tfV8/fr1RpK54IILjMfjsdqXLVtmJJl58+ZZbW3btjXx8fHn3OfZ+hYfH2/atm1rPV+xYoWRZP72t7951Q0aNMj4+PiYvXv3Wm2SjJ+fn1fbl19+aSSZF154odR7nWru3LlGknnttdestsLCQuN2u01QUJDXsbdt29bExcWddX+n9kmS+eyzz6y277//3vj7+5vbbrvNaiv5Hg8ZMsTr9RkZGUaSGTVqlFf7X/7yFyPJpKamevVLktm8ebPVtmbNGiPJBAQEmO+//95qf/nll40ks379eqstPj7eSDL333+/1VZcXGzi4uKMn5+f9fP2008/GUnm8ccfL9cYlPwMlbxXcXGx6dChg4mNjTXFxcVW3dGjR01kZKS57rrrSo3LPffc47XP2267zTRv3tx6np6ebiSZxMREr7rhw4eX6uvs2bONJJOZmVmqr+X9GXK5XCYhIaFcxw9wCgn1RlBQ0FlXI5VcePjee+9V+oJXh8OhESNGlLv+7rvvVpMmTazngwYNUqtWrfThhx9W6v3L68MPP1SDBg30wAMPeLU/9NBDMsZo1apVXu0xMTG68MILrefdunWT0+nUd999d873CQsL05AhQ6y2Ro0a6YEHHlB+fr42btxY6WNwu92Kjo62nrdp00a33HKL1qxZo6KiIq/aMWPGlOqXJE2YMMGrvWT2Jzk52as9KipKbrfbet6rVy9Jv52ebNOmTan2ssbl1GXEJafACgsL9dFHH53jSMsnIyNDe/bs0Z133qlffvlFP//8s37++WcdOXJEffv21aZNm0r9XJ8+LldeeaV++eUXeTweSbJO8ZTMMJW4//77K9y/8vwMBQcHa8uWLTpw4ECF94/6hwCDeiM/P98rLJzuz3/+sy6//HKNGjVKoaGhGjx4sJYtW1ahMHPBBRdU6ILdDh06eD338fFR+/btz3r9R1X4/vvvFR4eXmo8OnfubG0/1al/pEs0bdpUv/766znfp0OHDl6nU872PhVx+thJ0h//+EcdPXpUP/30k1d7ZGRkqX75+vqqffv2Xu1hYWEKDg4+5/G7XC5JUkRERJntp4+Lr6+v/vCHP5Tqq6Qq+17v2bNHkhQfH6+WLVt6PRYtWqSCggLl5eV5veb042ratKlX/0vG6fTxO33cyqM8P0OzZs3Sjh07FBERocsuu0zTpk07Z0hG/cU1MKgX/vvf/yovL++s//AGBARo06ZNWr9+vZKTk7V69WotXbpU1157rdauXasGDRqc830qct1KeZ3pZntFRUXl6lNVONP7mNMu+K2tzvR9Ke+NDM90/LVpXEqC9uzZs8+4xD8oKMjr+fnsf3ne64477tCVV16pd999V2vXrtXs2bP197//Xe+884769+9f5X2CvTEDg3rhX//6lyQpNjb2rHW+vr7q27evnnvuOX399dd66qmnlJqaqvXr10sq/x+88ir5X3MJY4z27t3rtWKoadOmZd4c7PRZgor0rW3btjpw4ECpU2q7du2ytleFtm3bas+ePaVmsarifU4fO0n69ttv1bhx43NeANy2bVsVFxeX2kdOTo5yc3Or7PhLFBcXl5pJ+PbbbyXJ+l7/3p+tktMzTqdTMTExZT4qupy+ZJwyMzO92vfu3Vuqtqp+N1q1aqX77rtPK1asUGZmppo3b66nnnqqSvaNuoUAgzovNTVVTz75pCIjIzV06NAz1h06dKhUW8n/ZEuW1pbcP6Sq7jb6f//3f14h4u2339aPP/7o9b/NCy+8UJ9++qkKCwuttpUrV5Zabl2Rvt1www0qKirSiy++6NU+Z84c+fj4VNn/dm+44QZlZ2dr6dKlVtvJkyf1wgsvKCgoSFdffXWl952Wlua1vHb//v1677331K9fv3POTN1www2SpLlz53q1P/fcc5JUapVNVTh1rI0xevHFF9WoUSP17dtXktS4cWNJlf/Zio6O1oUXXqhnnnlG+fn5pbafflqtPEoC//z5873aX3jhhVK1v/d3o6ioqNQprpCQEIWHh5da2g5InEJCHbNq1Srt2rVLJ0+eVE5OjlJTU5WSkqK2bdvq/fffl7+//xlfO336dG3atElxcXFq27atDh48qPnz56t169bWvUMuvPBCBQcHa+HChWrSpIkCAwPVq1evUtcIlFezZs10xRVXaMSIEcrJydHcuXPVvn17r6Xeo0aN0ttvv63rr79ed9xxh/7zn//otdde87ogsqJ9u+mmm9SnTx89+uij2rdvn7p37661a9fqvffeU2JiYql9V9bo0aP18ssva/jw4UpPT1e7du309ttv65NPPtHcuXPPek3SuXTp0kWxsbFey6glWXezPZvu3bsrPj5er7zyinJzc3X11Vdr69atWrJkiW699Vb16dOn0v0qi7+/v1avXq34+Hj16tVLq1atUnJysh555BFrtiggIEBRUVFaunSp/vjHP6pZs2bq0qVLqSXhZ+Lr66tFixapf//+uuiiizRixAhdcMEF+uGHH7R+/Xo5nU598MEHFep3dHS0Bg4cqLlz5+qXX36xllGXzB6dOutSckH1o48+qsGDB6tRo0a66aabyn3TwMOHD6t169YaNGiQunfvrqCgIH300Ufatm2bnn322Qr1G/VEDa6AAqpMyTLqkoefn58JCwsz1113nZk3b57Xct0Spy+jXrdunbnllltMeHi48fPzM+Hh4WbIkCHm22+/9Xrde++9Z6KiokzDhg29li1fffXV5qKLLiqzf2daRv3mm2+aKVOmmJCQEBMQEGDi4uK8luWWePbZZ80FF1xgHA6Hufzyy81nn31Wap9n69vpy6iNMebw4cNm/PjxJjw83DRq1Mh06NDBzJ4922sJrjG/LYEta2nrmZZ3ny4nJ8eMGDHCtGjRwvj5+ZmuXbuWudS7osuoExISzGuvvWY6dOhgHA6HueSSS7yWLxvzv+9xWUvjT5w4YZ544gkTGRlpGjVqZCIiIsyUKVPM8ePHy9WvssYlMzPTSDKzZ8+22uLj401gYKD5z3/+Y/r162caN25sQkNDzeOPP26Kioq8Xr9582YTHR1t/Pz8zrmk+vRl1CW++OILM2DAANO8eXPjcDhM27ZtzR133GHWrVt3znEp+T06dSn0kSNHTEJCgmnWrJkJCgoyt956q9m9e7eRZGbOnOn1+ieffNJccMEFxtfX12s/5fkZKigoMBMnTjTdu3c3TZo0MYGBgaZ79+5m/vz5ZxwD1G8+xtjkKjwA+P/5+PgoISGh1Cmw2mj48OF6++23yzytY1cZGRm65JJL9Nprr531tCxQnbgGBgBwRseOHSvVNnfuXPn6+uqqq66qgR4Bv+EaGADAGc2aNUvp6enq06ePGjZsqFWrVmnVqlUaPXp0qfvgAOcTAQYAcEZ/+tOflJKSoieffFL5+flq06aNpk2bpkcffbSmu4Z6jmtgAACA7XANDAAAsB0CDAAAsJ06ew1McXGxDhw4oCZNmlT57d8BAED1MMbo8OHDCg8PL/VBsKeqswHmwIEDXCEPAIBN7d+/X61btz7j9jobYEpuUb5//345nc4a7k09deSIFB7+29cHDkjlvKU4AKD+8ng8ioiIOOdHjdTZAFNy2sjpdBJgasqpH6jndBJgAADldq7LP7iIFwAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2E7Dmu5AfdJucrLX830z42qoJwAA2BszMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYqFGDatWsnHx+fUo+EhARJ0vHjx5WQkKDmzZsrKChIAwcOVE5Ojtc+srKyFBcXp8aNGyskJEQTJ07UyZMnvWo2bNigHj16yOFwqH379kpKSvp9RwkAAOqUCgWYbdu26ccff7QeKSkpkqTbb79dkjR+/Hh98MEHWr58uTZu3KgDBw5owIAB1uuLiooUFxenwsJCbd68WUuWLFFSUpKmTp1q1WRmZiouLk59+vRRRkaGEhMTNWrUKK1Zs6YqjhcAANQBPsYYU9kXJyYmauXKldqzZ488Ho9atmypN954Q4MGDZIk7dq1S507d1ZaWpp69+6tVatW6cYbb9SBAwcUGhoqSVq4cKEmTZqkn376SX5+fpo0aZKSk5O1Y8cO630GDx6s3NxcrV69utx983g8crlcysvLk9PprOwhVql2k5O9nu+bGVdDPTlPjhyRgoJ++zo/XwoMrNn+AABqvfL+/a70NTCFhYV67bXXdM8998jHx0fp6ek6ceKEYmJirJpOnTqpTZs2SktLkySlpaWpa9euVniRpNjYWHk8Hu3cudOqOXUfJTUl+ziTgoICeTwerwcAAKibKh1gVqxYodzcXA0fPlySlJ2dLT8/PwUHB3vVhYaGKjs726o5NbyUbC/ZdrYaj8ejY8eOnbE/M2bMkMvlsh4RERGVPTQAAFDLVTrA/OMf/1D//v0VHh5elf2ptClTpigvL8967N+/v6a7BAAAqknDyrzo+++/10cffaR33nnHagsLC1NhYaFyc3O9ZmFycnIUFhZm1WzdutVrXyWrlE6tOX3lUk5OjpxOpwICAs7YJ4fDIYfDUZnDAQAANlOpGZjFixcrJCREcXH/uwg1OjpajRo10rp166y23bt3KysrS263W5Lkdru1fft2HTx40KpJSUmR0+lUVFSUVXPqPkpqSvYBAABQ4QBTXFysxYsXKz4+Xg0b/m8Cx+VyaeTIkZowYYLWr1+v9PR0jRgxQm63W71795Yk9evXT1FRURo2bJi+/PJLrVmzRo899pgSEhKs2ZMxY8bou+++08MPP6xdu3Zp/vz5WrZsmcaPH19FhwwAAOyuwqeQPvroI2VlZemee+4ptW3OnDny9fXVwIEDVVBQoNjYWM2fP9/a3qBBA61cuVJjx46V2+1WYGCg4uPjNX36dKsmMjJSycnJGj9+vObNm6fWrVtr0aJFio2NreQhAgCAuuZ33QemNuM+MLUA94EBAFRQtd8HBgAAoKYQYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO1UOMD88MMPuuuuu9S8eXMFBASoa9eu+uyzz6ztxhhNnTpVrVq1UkBAgGJiYrRnzx6vfRw6dEhDhw6V0+lUcHCwRo4cqfz8fK+ar776SldeeaX8/f0VERGhWbNmVfIQa0a7ycmlHgAAoGpUKMD8+uuvuvzyy9WoUSOtWrVKX3/9tZ599lk1bdrUqpk1a5aef/55LVy4UFu2bFFgYKBiY2N1/Phxq2bo0KHauXOnUlJStHLlSm3atEmjR4+2tns8HvXr109t27ZVenq6Zs+erWnTpumVV16pgkMGAAB252OMMeUtnjx5sj755BP9+9//LnO7MUbh4eF66KGH9Je//EWSlJeXp9DQUCUlJWnw4MH65ptvFBUVpW3btqlnz56SpNWrV+uGG27Qf//7X4WHh2vBggV69NFHlZ2dLT8/P+u9V6xYoV27dpWrrx6PRy6XS3l5eXI6neU9xCpTnhmXfTPjzkNPatCRI1JQ0G9f5+dLgYE12x8AQK1X3r/fFZqBef/999WzZ0/dfvvtCgkJ0SWXXKJXX33V2p6Zmans7GzFxMRYbS6XS7169VJaWpokKS0tTcHBwVZ4kaSYmBj5+vpqy5YtVs1VV11lhRdJio2N1e7du/Xrr7+W2beCggJ5PB6vBwAAqJsqFGC+++47LViwQB06dNCaNWs0duxYPfDAA1qyZIkkKTs7W5IUGhrq9brQ0FBrW3Z2tkJCQry2N2zYUM2aNfOqKWsfp77H6WbMmCGXy2U9IiIiKnJoAADARioUYIqLi9WjRw89/fTTuuSSSzR69Gjde++9WrhwYXX1r9ymTJmivLw867F///6a7hIAAKgmFQowrVq1UlRUlFdb586dlZWVJUkKCwuTJOXk5HjV5OTkWNvCwsJ08OBBr+0nT57UoUOHvGrK2sep73E6h8Mhp9Pp9QAAAHVThQLM5Zdfrt27d3u1ffvtt2rbtq0kKTIyUmFhYVq3bp213ePxaMuWLXK73ZIkt9ut3NxcpaenWzWpqakqLi5Wr169rJpNmzbpxIkTVk1KSoo6duzoteIJAADUTxUKMOPHj9enn36qp59+Wnv37tUbb7yhV155RQkJCZIkHx8fJSYm6m9/+5vef/99bd++XXfffbfCw8N16623Svptxub666/Xvffeq61bt+qTTz7RuHHjNHjwYIWHh0uS7rzzTvn5+WnkyJHauXOnli5dqnnz5mnChAlVe/QAAMCWGlak+NJLL9W7776rKVOmaPr06YqMjNTcuXM1dOhQq+bhhx/WkSNHNHr0aOXm5uqKK67Q6tWr5e/vb9W8/vrrGjdunPr27StfX18NHDhQzz//vLXd5XJp7dq1SkhIUHR0tFq0aKGpU6d63SsGAADUXxW6D4ydcB+YWoD7wAAAKqha7gMDAABQGxBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7TSs6Q7UZ+0mJ5dq2zczrgZ6AgCAvTADAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbKdCAWbatGny8fHxenTq1Mnafvz4cSUkJKh58+YKCgrSwIEDlZOT47WPrKwsxcXFqXHjxgoJCdHEiRN18uRJr5oNGzaoR48ecjgcat++vZKSkip/hAAAoM6p8AzMRRddpB9//NF6fPzxx9a28ePH64MPPtDy5cu1ceNGHThwQAMGDLC2FxUVKS4uToWFhdq8ebOWLFmipKQkTZ061arJzMxUXFyc+vTpo4yMDCUmJmrUqFFas2bN7zxUAABQVzSs8AsaNlRYWFip9ry8PP3jH//QG2+8oWuvvVaStHjxYnXu3FmffvqpevfurbVr1+rrr7/WRx99pNDQUF188cV68sknNWnSJE2bNk1+fn5auHChIiMj9eyzz0qSOnfurI8//lhz5sxRbGzs7zxcAABQF1R4BmbPnj0KDw/XH/7wBw0dOlRZWVmSpPT0dJ04cUIxMTFWbadOndSmTRulpaVJktLS0tS1a1eFhoZaNbGxsfJ4PNq5c6dVc+o+SmpK9nEmBQUF8ng8Xg8AAFA3VSjA9OrVS0lJSVq9erUWLFigzMxMXXnllTp8+LCys7Pl5+en4OBgr9eEhoYqOztbkpSdne0VXkq2l2w7W43H49GxY8fO2LcZM2bI5XJZj4iIiIocGgAAsJEKnULq37+/9XW3bt3Uq1cvtW3bVsuWLVNAQECVd64ipkyZogkTJljPPR4PIQYAgDrqdy2jDg4O1h//+Eft3btXYWFhKiwsVG5urldNTk6Odc1MWFhYqVVJJc/PVeN0Os8akhwOh5xOp9cDAADUTb8rwOTn5+s///mPWrVqpejoaDVq1Ejr1q2ztu/evVtZWVlyu92SJLfbre3bt+vgwYNWTUpKipxOp6KioqyaU/dRUlOyDwAAgAoFmL/85S/auHGj9u3bp82bN+u2225TgwYNNGTIELlcLo0cOVITJkzQ+vXrlZ6erhEjRsjtdqt3796SpH79+ikqKkrDhg3Tl19+qTVr1uixxx5TQkKCHA6HJGnMmDH67rvv9PDDD2vXrl2aP3++li1bpvHjx1f90QMAAFuq0DUw//3vfzVkyBD98ssvatmypa644gp9+umnatmypSRpzpw58vX11cCBA1VQUKDY2FjNnz/fen2DBg20cuVKjR07Vm63W4GBgYqPj9f06dOtmsjISCUnJ2v8+PGaN2+eWrdurUWLFrGEGgAAWHyMMaamO1EdPB6PXC6X8vLyauR6mHaTkyv1un0z46q4JzXoyBEpKOi3r/PzpcDAmu0PAKDWK+/fbz4LCQAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2E7Dmu5AXdFucnJNdwEAgHqDGRgAAGA7BBgAAGA7BBgAAGA7BBgAAGA7BBgAAGA7BBgAAGA7vyvAzJw5Uz4+PkpMTLTajh8/roSEBDVv3lxBQUEaOHCgcnJyvF6XlZWluLg4NW7cWCEhIZo4caJOnjzpVbNhwwb16NFDDodD7du3V1JS0u/pKgAAqEMqHWC2bduml19+Wd26dfNqHz9+vD744AMtX75cGzdu1IEDBzRgwABre1FRkeLi4lRYWKjNmzdryZIlSkpK0tSpU62azMxMxcXFqU+fPsrIyFBiYqJGjRqlNWvWVLa7AACgDqlUgMnPz9fQoUP16quvqmnTplZ7Xl6e/vGPf+i5557Ttddeq+joaC1evFibN2/Wp59+Kklau3atvv76a7322mu6+OKL1b9/fz355JN66aWXVFhYKElauHChIiMj9eyzz6pz584aN26cBg0apDlz5lTBIQMAALurVIBJSEhQXFycYmJivNrT09N14sQJr/ZOnTqpTZs2SktLkySlpaWpa9euCg0NtWpiY2Pl8Xi0c+dOq+b0fcfGxlr7KEtBQYE8Ho/XAwAA1E0V/iiBt956S59//rm2bdtWalt2drb8/PwUHBzs1R4aGqrs7Gyr5tTwUrK9ZNvZajwej44dO6aAgIBS7z1jxgw98cQTFT0cAABgQxWagdm/f78efPBBvf766/L396+uPlXKlClTlJeXZz32799f010CAADVpEIzMOnp6Tp48KB69OhhtRUVFWnTpk168cUXtWbNGhUWFio3N9drFiYnJ0dhYWGSpLCwMG3dutVrvyWrlE6tOX3lUk5OjpxOZ5mzL5LkcDjkcDgqcji10ukfCrlvZlwN9QQAgNqrQgGmb9++2r59u1fbiBEj1KlTJ02aNEkRERFq1KiR1q1bp4EDB0qSdu/eraysLLndbkmS2+3WU089pYMHDyokJESSlJKSIqfTqaioKKvmww8/9HqflJQUax/1SVmfck2oAQDUdxUKME2aNFGXLl282gIDA9W8eXOrfeTIkZowYYKaNWsmp9Op+++/X263W71795Yk9evXT1FRURo2bJhmzZql7OxsPfbYY0pISLBmUMaMGaMXX3xRDz/8sO655x6lpqZq2bJlSk4u/cccAADUPxW+iPdc5syZI19fXw0cOFAFBQWKjY3V/Pnzre0NGjTQypUrNXbsWLndbgUGBio+Pl7Tp0+3aiIjI5WcnKzx48dr3rx5at26tRYtWqTY2Niq7i4AALAhH2OMqelOVAePxyOXy6W8vDw5nc5qf7+yTvVUF9ucQjpyRAoK+u3r/HwpMLBm+wMAqPXK+/ebz0ICAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC207CmO4CKazc52ev5vplxNdQTAABqBjMwAADAdggwAADAdggwAADAdggwAADAdriItw44/aJeiQt7AQB1GzMwAADAdggwAADAdggwAADAdggwAADAdggwAADAdggwAADAdggwAADAdggwAADAdggwAADAdggwAADAdggwAADAdggwAADAdggwAADAdioUYBYsWKBu3brJ6XTK6XTK7XZr1apV1vbjx48rISFBzZs3V1BQkAYOHKicnByvfWRlZSkuLk6NGzdWSEiIJk6cqJMnT3rVbNiwQT169JDD4VD79u2VlJRU+SMEAAB1ToUCTOvWrTVz5kylp6frs88+07XXXqtbbrlFO3fulCSNHz9eH3zwgZYvX66NGzfqwIEDGjBggPX6oqIixcXFqbCwUJs3b9aSJUuUlJSkqVOnWjWZmZmKi4tTnz59lJGRocTERI0aNUpr1qypokMGAAB252OMMb9nB82aNdPs2bM1aNAgtWzZUm+88YYGDRokSdq1a5c6d+6stLQ09e7dW6tWrdKNN96oAwcOKDQ0VJK0cOFCTZo0ST/99JP8/Pw0adIkJScna8eOHdZ7DB48WLm5uVq9enW5++XxeORyuZSXlyen0/l7DrFc2k1Orvb3qIh9M+NqugvSkSNSUNBvX+fnS4GBNdsfAECtV96/35W+BqaoqEhvvfWWjhw5IrfbrfT0dJ04cUIxMTFWTadOndSmTRulpaVJktLS0tS1a1crvEhSbGysPB6PNYuTlpbmtY+SmpJ9nElBQYE8Ho/XAwAA1E0VDjDbt29XUFCQHA6HxowZo3fffVdRUVHKzs6Wn5+fgoODvepDQ0OVnZ0tScrOzvYKLyXbS7adrcbj8ejYsWNn7NeMGTPkcrmsR0REREUPDQAA2ESFA0zHjh2VkZGhLVu2aOzYsYqPj9fXX39dHX2rkClTpigvL8967N+/v6a7BAAAqknDir7Az89P7du3lyRFR0dr27Ztmjdvnv785z+rsLBQubm5XrMwOTk5CgsLkySFhYVp69atXvsrWaV0as3pK5dycnLkdDoVEBBwxn45HA45HI6KHg4AALCh330fmOLiYhUUFCg6OlqNGjXSunXrrG27d+9WVlaW3G63JMntdmv79u06ePCgVZOSkiKn06moqCir5tR9lNSU7AMAAKBCMzBTpkxR//791aZNGx0+fFhvvPGGNmzYoDVr1sjlcmnkyJGaMGGCmjVrJqfTqfvvv19ut1u9e/eWJPXr109RUVEaNmyYZs2apezsbD322GNKSEiwZk/GjBmjF198UQ8//LDuuecepaamatmyZUpOrl2rfAAAQM2pUIA5ePCg7r77bv34449yuVzq1q2b1qxZo+uuu06SNGfOHPn6+mrgwIEqKChQbGys5s+fb72+QYMGWrlypcaOHSu3263AwEDFx8dr+vTpVk1kZKSSk5M1fvx4zZs3T61bt9aiRYsUGxtbRYcMAADs7nffB6a24j4w3AcGAGA/1X4fGAAAgJpS4VVIsIfTZ4RqxYwMAABVhBkYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOw1rugM4P9pNTi7Vtm9mXA30BACA348ZGAAAYDsEGAAAYDsEGAAAYDsVCjAzZszQpZdeqiZNmigkJES33nqrdu/e7VVz/PhxJSQkqHnz5goKCtLAgQOVk5PjVZOVlaW4uDg1btxYISEhmjhxok6ePOlVs2HDBvXo0UMOh0Pt27dXUlJS5Y4QAADUORUKMBs3blRCQoI+/fRTpaSk6MSJE+rXr5+OHDli1YwfP14ffPCBli9fro0bN+rAgQMaMGCAtb2oqEhxcXEqLCzU5s2btWTJEiUlJWnq1KlWTWZmpuLi4tSnTx9lZGQoMTFRo0aN0po1a6rgkAEAgN35GGNMZV/8008/KSQkRBs3btRVV12lvLw8tWzZUm+88YYGDRokSdq1a5c6d+6stLQ09e7dW6tWrdKNN96oAwcOKDQ0VJK0cOFCTZo0ST/99JP8/Pw0adIkJScna8eOHdZ7DR48WLm5uVq9enW5+ubxeORyuZSXlyen01nZQyy3slb51HbVvgrpyBEpKOi3r/PzpcDA6n0/AIDtlffv9++6BiYvL0+S1KxZM0lSenq6Tpw4oZiYGKumU6dOatOmjdLS0iRJaWlp6tq1qxVeJCk2NlYej0c7d+60ak7dR0lNyT7KUlBQII/H4/UAAAB1U6UDTHFxsRITE3X55ZerS5cukqTs7Gz5+fkpODjYqzY0NFTZ2dlWzanhpWR7ybaz1Xg8Hh07dqzM/syYMUMul8t6REREVPbQAABALVfpAJOQkKAdO3borbfeqsr+VNqUKVOUl5dnPfbv31/TXQIAANWkUnfiHTdunFauXKlNmzapdevWVntYWJgKCwuVm5vrNQuTk5OjsLAwq2br1q1e+ytZpXRqzekrl3JycuR0OhUQEFBmnxwOhxwOR2UOBwAA2EyFZmCMMRo3bpzeffddpaamKjIy0mt7dHS0GjVqpHXr1lltu3fvVlZWltxutyTJ7XZr+/btOnjwoFWTkpIip9OpqKgoq+bUfZTUlOwDAADUbxWagUlISNAbb7yh9957T02aNLGuWXG5XAoICJDL5dLIkSM1YcIENWvWTE6nU/fff7/cbrd69+4tSerXr5+ioqI0bNgwzZo1S9nZ2XrssceUkJBgzaCMGTNGL774oh5++GHdc889Sk1N1bJly5ScbL+VPgAAoOpVaAZmwYIFysvL0zXXXKNWrVpZj6VLl1o1c+bM0Y033qiBAwfqqquuUlhYmN555x1re4MGDbRy5Uo1aNBAbrdbd911l+6++25Nnz7dqomMjFRycrJSUlLUvXt3Pfvss1q0aJFiY2Or4JABAIDd/a77wNRm3Afm3LgPDACgtjkv94EBAACoCQQYAABgO5VaRo26oazTXtV+WgkAgCrADAwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALCdhjXdAdQu7SYnez3fNzOuhnoCAMCZMQMDAABshwADAABshwADAABshwADAABshwADAABshwADAABshwADAABshwADAABshwADAABshwADAABshwADAABsp8IBZtOmTbrpppsUHh4uHx8frVixwmu7MUZTp05Vq1atFBAQoJiYGO3Zs8er5tChQxo6dKicTqeCg4M1cuRI5efne9V89dVXuvLKK+Xv76+IiAjNmjWr4kcHAADqpAoHmCNHjqh79+566aWXytw+a9YsPf/881q4cKG2bNmiwMBAxcbG6vjx41bN0KFDtXPnTqWkpGjlypXatGmTRo8ebW33eDzq16+f2rZtq/T0dM2ePVvTpk3TK6+8UolDBAAAdU2FP426f//+6t+/f5nbjDGaO3euHnvsMd1yyy2SpP/7v/9TaGioVqxYocGDB+ubb77R6tWrtW3bNvXs2VOS9MILL+iGG27QM888o/DwcL3++usqLCzUP//5T/n5+emiiy5SRkaGnnvuOa+gAwAA6qcqvQYmMzNT2dnZiomJsdpcLpd69eqltLQ0SVJaWpqCg4Ot8CJJMTEx8vX11ZYtW6yaq666Sn5+flZNbGysdu/erV9//bXM9y4oKJDH4/F6AACAuqlKA0x2drYkKTQ01Ks9NDTU2padna2QkBCv7Q0bNlSzZs28asrax6nvcboZM2bI5XJZj4iIiN9/QAAAoFaqM6uQpkyZory8POuxf//+mu4SAACoJhW+BuZswsLCJEk5OTlq1aqV1Z6Tk6OLL77Yqjl48KDX606ePKlDhw5Zrw8LC1NOTo5XTcnzkprTORwOORyOKjkO/E+7ycml2vbNjKuBngAA8D9VGmAiIyMVFhamdevWWYHF4/Foy5YtGjt2rCTJ7XYrNzdX6enpio6OliSlpqaquLhYvXr1smoeffRRnThxQo0aNZIkpaSkqGPHjmratGlVdrlSyvqjDgAAzp8Kn0LKz89XRkaGMjIyJP124W5GRoaysrLk4+OjxMRE/e1vf9P777+v7du36+6771Z4eLhuvfVWSVLnzp11/fXX695779XWrVv1ySefaNy4cRo8eLDCw8MlSXfeeaf8/Pw0cuRI7dy5U0uXLtW8efM0YcKEKjtwAABgXxWegfnss8/Up08f63lJqIiPj1dSUpIefvhhHTlyRKNHj1Zubq6uuOIKrV69Wv7+/tZrXn/9dY0bN059+/aVr6+vBg4cqOeff97a7nK5tHbtWiUkJCg6OlotWrTQ1KlTWUINAAAkST7GGFPTnagOHo9HLpdLeXl5cjqdVbrv+n4KqdzXwBw5IgUF/fZ1fr4UGFh9nQIA1Anl/ftdZ1YhAQCA+qNKL+JF/XD6DBSrkgAA5xszMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHb4MEf8bqd/uKPEBzwCAKoXMzAAAMB2CDAAAMB2CDAAAMB2CDAAAMB2uIgX1aLd5GQFFB7XNzXdEQBAncQMDAAAsB0CDAAAsB0CDAAAsB0CDAAAsB0u4sV50fmvq3XMz9+rjbv1AgAqixkYAABgOwQYAABgO5xCQo05/UMgOaUEACgvZmAAAIDtMAODWuP0GRmJWRkAQNkIMKjVOM0EACgLp5AAAIDtMAMDWynrNNPpmKUBgLqPGRgAAGA7zMCgzmGWBgDqvlo9A/PSSy+pXbt28vf3V69evbR169aa7hIAAKgFau0MzNKlSzVhwgQtXLhQvXr10ty5cxUbG6vdu3crJCSkprsHmyvPLE1ZmLkBgNqh1gaY5557Tvfee69GjBghSVq4cKGSk5P1z3/+U5MnT67h3qG+4vQUANQOtTLAFBYWKj09XVOmTLHafH19FRMTo7S0tDJfU1BQoIKCAut5Xl6eJMnj8VR5/4oLjlb5PuuiosLjKhn9ooKjKjbFNdqf86XN+OVVsp8dT8Ses6bL42vO+Zry1ABAbVHyd9sYc9a6Whlgfv75ZxUVFSk0NNSrPTQ0VLt27SrzNTNmzNATTzxRqj0iIqJa+ojycZV8Mf/umuyGLbnmVs9rKrNfADjfDh8+LJfLdcbttTLAVMaUKVM0YcIE63lxcbEOHTqk5s2by8fHx6vW4/EoIiJC+/fvl9PpPN9drdUYm7NjfM6MsTk7xufMGJuzq2/jY4zR4cOHFR4efta6WhlgWrRooQYNGignJ8erPScnR2FhYWW+xuFwyOFweLUFBwef9X2cTme9+GGoDMbm7BifM2Nszo7xOTPG5uzq0/icbealRK1cRu3n56fo6GitW7fOaisuLta6devkdrtrsGcAAKA2qJUzMJI0YcIExcfHq2fPnrrssss0d+5cHTlyxFqVBAAA6q9aG2D+/Oc/66efftLUqVOVnZ2tiy++WKtXry51YW9lOBwOPf7446VOOYGxORfG58wYm7NjfM6MsTk7xqdsPuZc65QAAABqmVp5DQwAAMDZEGAAAIDtEGAAAIDtEGAAAIDtEGAAAIDt1MsA89JLL6ldu3by9/dXr169tHXr1pruUrWaMWOGLr30UjVp0kQhISG69dZbtXv3bq+a48ePKyEhQc2bN1dQUJAGDhxY6k7IWVlZiouLU+PGjRUSEqKJEyfq5MmT5/NQzouZM2fKx8dHiYmJVlt9Hp8ffvhBd911l5o3b66AgAB17dpVn332mbXdGKOpU6eqVatWCggIUExMjPbs2eO1j0OHDmno0KFyOp0KDg7WyJEjlZ+ff74PpcoVFRXpr3/9qyIjIxUQEKALL7xQTz75pNeH0NWX8dm0aZNuuukmhYeHy8fHRytWrPDaXlXj8NVXX+nKK6+Uv7+/IiIiNGvWrOo+tCpxtvE5ceKEJk2apK5duyowMFDh4eG6++67deDAAa991OXxqRRTz7z11lvGz8/P/POf/zQ7d+409957rwkODjY5OTk13bVqExsbaxYvXmx27NhhMjIyzA033GDatGlj8vPzrZoxY8aYiIgIs27dOvPZZ5+Z3r17mz/96U/W9pMnT5ouXbqYmJgY88UXX5gPP/zQtGjRwkyZMqUmDqnabN261bRr185069bNPPjgg1Z7fR2fQ4cOmbZt25rhw4ebLVu2mO+++86sWbPG7N2716qZOXOmcblcZsWKFebLL780N998s4mMjDTHjh2zaq6//nrTvXt38+mnn5p///vfpn379mbIkCE1cUhV6qmnnjLNmzc3K1euNJmZmWb58uUmKCjIzJs3z6qpL+Pz4YcfmkcffdS88847RpJ59913vbZXxTjk5eWZ0NBQM3ToULNjxw7z5ptvmoCAAPPyyy+fr8OstLONT25uromJiTFLly41u3btMmlpaeayyy4z0dHRXvuoy+NTGfUuwFx22WUmISHBel5UVGTCw8PNjBkzarBX59fBgweNJLNx40ZjzG+/PI0aNTLLly+3ar755hsjyaSlpRljfvvl8/X1NdnZ2VbNggULjNPpNAUFBef3AKrJ4cOHTYcOHUxKSoq5+uqrrQBTn8dn0qRJ5oorrjjj9uLiYhMWFmZmz55tteXm5hqHw2HefPNNY4wxX3/9tZFktm3bZtWsWrXK+Pj4mB9++KH6On8exMXFmXvuucerbcCAAWbo0KHGmPo7Pqf/ga6qcZg/f75p2rSp1+/UpEmTTMeOHav5iKpWWQHvdFu3bjWSzPfff2+MqV/jU1716hRSYWGh0tPTFRMTY7X5+voqJiZGaWlpNdiz8ysvL0+S1KxZM0lSenq6Tpw44TUunTp1Ups2baxxSUtLU9euXb3uhBwbGyuPx6OdO3eex95Xn4SEBMXFxXmNg1S/x+f9999Xz549dfvttyskJESXXHKJXn31VWt7ZmamsrOzvcbG5XKpV69eXmMTHBysnj17WjUxMTHy9fXVli1bzt/BVIM//elPWrdunb799ltJ0pdffqmPP/5Y/fv3l8T4lKiqcUhLS9NVV10lPz8/qyY2Nla7d+/Wr7/+ep6O5vzIy8uTj4+P9aHEjE9ptfajBKrDzz//rKKiolIfRxAaGqpdu3bVUK/Or+LiYiUmJuryyy9Xly5dJEnZ2dny8/Mr9endoaGhys7OtmrKGreSbXb31ltv6fPPP9e2bdtKbavP4/Pdd99pwYIFmjBhgh555BFt27ZNDzzwgPz8/BQfH28dW1nHfurYhISEeG1v2LChmjVrZuuxkaTJkyfL4/GoU6dOatCggYqKivTUU09p6NChklTvx6dEVY1Ddna2IiMjS+2jZFvTpk2rpf/n2/HjxzVp0iQNGTLE+vRpxqe0ehVg8Nssw44dO/Txxx/XdFdqjf379+vBBx9USkqK/P39a7o7tUpxcbF69uypp59+WpJ0ySWXaMeOHVq4cKHi4+NruHc1b9myZXr99df1xhtv6KKLLlJGRoYSExMVHh7O+KBSTpw4oTvuuEPGGC1YsKCmu1Or1atTSC1atFCDBg1KrR7JyclRWFhYDfXq/Bk3bpxWrlyp9evXq3Xr1lZ7WFiYCgsLlZub61V/6riEhYWVOW4l2+wsPT1dBw8eVI8ePdSwYUM1bNhQGzdu1PPPP6+GDRsqNDS03o5Pq1atFBUV5dXWuXNnZWVlSfrfsZ3tdyosLEwHDx702n7y5EkdOnTI1mMjSRMnTtTkyZM1ePBgde3aVcOGDdP48eM1Y8YMSYxPiaoah7r6e1aiJLx8//33SklJsWZfJManLPUqwPj5+Sk6Olrr1q2z2oqLi7Vu3Tq53e4a7Fn1MsZo3Lhxevfdd5WamlpqijE6OlqNGjXyGpfdu3crKyvLGhe3263t27d7/QKV/IKd/gfObvr27avt27crIyPDevTs2VNDhw61vq6v43P55ZeXWnL/7bffqm3btpKkyMhIhYWFeY2Nx+PRli1bvMYmNzdX6enpVk1qaqqKi4vVq1ev83AU1efo0aPy9fX+Z7RBgwYqLi6WxPiUqKpxcLvd2rRpk06cOGHVpKSkqGPHjrY/PVISXvbs2aOPPvpIzZs399pe38enTDV9FfH59tZbbxmHw2GSkpLM119/bUaPHm2Cg4O9Vo/UNWPHjjUul8ts2LDB/Pjjj9bj6NGjVs2YMWNMmzZtTGpqqvnss8+M2+02brfb2l6yTLhfv34mIyPDrF692rRs2dL2y4TP5NRVSMbU3/HZunWradiwoXnqqafMnj17zOuvv24aN25sXnvtNatm5syZJjg42Lz33nvmq6++MrfcckuZy2MvueQSs2XLFvPxxx+bDh062G6ZcFni4+PNBRdcYC2jfuedd0yLFi3Mww8/bNXUl/E5fPiw+eKLL8wXX3xhJJnnnnvOfPHFF9YqmqoYh9zcXBMaGmqGDRtmduzYYd566y3TuHFjWywTPtv4FBYWmptvvtm0bt3aZGRkeP07feqKoro8PpVR7wKMMca88MILpk2bNsbPz89cdtll5tNPP63pLlUrSWU+Fi9ebNUcO3bM3HfffaZp06amcePG5rbbbjM//vij13727dtn+vfvbwICAkyLFi3MQw89ZE6cOHGej+b8OD3A1Ofx+eCDD0yXLl2Mw+EwnTp1Mq+88orX9uLiYvPXv/7VhIaGGofDYfr27Wt2797tVfPLL7+YIUOGmKCgION0Os2IESPM4cOHz+dhVAuPx2MefPBB06ZNG+Pv72/+8Ic/mEcffdTrj059GZ/169eX+e9MfHy8MabqxuHLL780V1xxhXE4HOaCCy4wM2fOPF+H+LucbXwyMzPP+O/0+vXrrX3U5fGpDB9jTrllJAAAgA3Uq2tgAABA3UCAAQAAtkOAAQAAtkOAAQAAtkOAAQAAtkOAAQAAtkOAAQAAtkOAAQAAtkOAAQAAtkOAAQAAtkOAAQAAtvP/AUEYNAunFIVYAAAAAElFTkSuQmCC",
+      "text/plain": [
+       "<Figure size 640x480 with 1 Axes>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "import matplotlib.pyplot as plt\n",
+    "\n",
+    "lens = [len(x[\"prompt\"][\"input_ids\"]) for x in data[\"train\"]]\n",
+    "plt.hist(lens, bins=100)\n",
+    "plt.title(\"Distribution of prompt lengths\")\n",
+    "plt.axvline(256, color=\"red\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "<matplotlib.lines.Line2D at 0x7f6eef316ce0>"
+      ]
+     },
+     "execution_count": 3,
+     "metadata": {},
+     "output_type": "execute_result"
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkUAAAGzCAYAAAAhXWNYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABagUlEQVR4nO3dd3xT9f4/8FeSNulMB12UllVG2WhZdeGoVOhVUVRElMpQ8RZZ9wLiAHHB1auCyhD5fcWLKMPNEMQC4iirUNnIKJu2lNKkLZ3J+/dHyaGhBTrShiSv5+ORB805n5zzOZ/kJC/O+XzOUYmIgIiIiMjFqe1dASIiIqIbAUMRERERERiKiIiIiAAwFBEREREBYCgiIiIiAsBQRERERASAoYiIiIgIAEMREREREQCGIiIiIiIADEUNbuPGjVCpVPj666/tXZVqyczMxCOPPIJGjRpBpVJh5syZVy2bn5+PESNGICwsDCqVCmPHjq3Rup5++mk0b97cappKpcJrr71W43rTZZbP3MaNG+1dFbur6jPm6g4dOoQ+ffrAz88PKpUK33//vb2rVKU777wTd955p72rcU01+b50BK+99hpUKpW9q3FdtvxddbNBfW44CxcuxNChQ6HT6XDkyBE0adLEav6dd96J7Oxs7Nmzx041dBzjxo3D2rVrMXXqVISFhaFbt25XLfv2229j4cKFePXVVxEVFYV27do1YE2JqDYSExORnp6Ot956C/7+/tfcx+vbvn37sGzZMocNrzX5vqSa+/LLL5GVlVXj/3DXhFOGIovi4mLMmDEDH330kb2r4rDWr1+PBx98EP/+97+rVbZXr16YOnVqA9SMiOqqsLAQKSkpePnllzFq1Ch7Vwf79u3DtGnTcOedd1YKRT///LN9KlUDNfm+pJr78ssvsWfPnnoNRU59+qxr16749NNPcebMGXtXpcEVFBTYZDlZWVnw9/e3eVm6PrPZjKKiIntXg5zYuXPnAMAh9lutVgutVmvvalwTvwMdn1OHopdeegkmkwkzZsy4Zrljx45BpVJh4cKFleZd2afFco7177//xpNPPgk/Pz8EBwfj1VdfhYjg5MmTePDBB6HX6xEWFob33nuvynWaTCa89NJLCAsLg7e3Nx544AGcPHmyUrktW7bgvvvug5+fH7y8vNC7d2/88ccfVmUsddq3bx+eeOIJBAQE4LbbbrvmNh89ehSPPvooAgMD4eXlhV69emHVqlXK/IULF0KlUkFEMHv2bKhUqqueW7acz01PT8eqVauUsseOHVOWc+zYsSpfU5N+Lhs2bIBKpcJ3331Xad6XX34JlUqFlJSUay4jNzcX48aNQ/PmzaHT6RAREYEhQ4YgOztbKZOVlYXhw4cjNDQUHh4e6NKlCz7//HNlfmlpKQIDAzF06NBKyzcajfDw8LD6n2JxcTGmTp2KVq1aQafTITIyEhMnTkRxcbHVa1UqFUaNGoXFixejQ4cO0Ol0WLNmDQDg9OnTGDZsGEJDQ6HT6dChQwf83//9X6X1nzp1Cv3794e3tzdCQkIwbty4Suu5luutp7CwENHR0YiOjkZhYaEyPScnB40bN8Ytt9wCk8kEANi1axeefvpptGzZEh4eHggLC8OwYcNw/vx5q3XWdZ+yfJaWLl1arX3qSmazGTNnzkSHDh3g4eGB0NBQPPfcc7hw4YJVue3btyM+Ph5BQUHw9PREixYtMGzYsOsu/4cffkBCQgLCw8Oh0+kQFRWFN954Q2kni0OHDmHAgAEICwuDh4cHIiIi8Pjjj8NgMFxz+b/99hseffRRNG3aVPl8jRs3zur9qcprr72GZs2aAQAmTJgAlUqlHJ252umrqvqYWD6333//PTp27Kh8biyf3YpOnz6N4cOHK23RokULPP/88ygpKcHChQvx6KOPAgDuuusu5XvE8h1RVZ+i6+2rwOXv9//+97+YP38+oqKioNPp0L17d2zbtu2abWRhy+/LK+s0e/ZstGzZEl5eXujTpw9OnjwJEcEbb7yBiIgIeHp64sEHH0ROTo7VMqrzudq/fz88PT0xZMgQq9f+/vvv0Gg0mDRpUrW2/0pffPEFYmJi4OnpicDAQDz++OOV9rU777wTHTt2xL59+3DXXXfBy8sLTZo0wTvvvFNpecePH8cDDzxg9b21du3aSu//qlWrcPz4caV9r/yMms1mvPXWW4iIiICHhwfuueceHD58uGYbJ07os88+EwCybds2GTZsmHh4eMjp06eV+b1795YOHTooz9PT0wWAfPbZZ5WWBUCmTp2qPJ86daoAkK5du8qgQYNkzpw5kpCQIADk/fffl7Zt28rzzz8vc+bMkVtvvVUAyK+//qq8fsOGDQJAOnXqJJ07d5b3339fXnzxRfHw8JA2bdrIxYsXlbLJycmi1WolNjZW3nvvPfnggw+kc+fOotVqZcuWLZXq1L59e3nwwQdlzpw5Mnv27Ku2T0ZGhoSGhoqvr6+8/PLL8v7770uXLl1ErVbLt99+KyIiR44ckUWLFgkAuffee2XRokWyaNGiqy5v0aJFEhQUJF27dlXK5ufnK+9Fenq61Wss7bBhwwZlWmJiojRr1uyq7W82myUyMlIGDBhQqQ79+vWTqKioq26ziEheXp507NhRNBqNPPPMMzJ37lx54403pHv37rJz504REbl48aK0a9dO3N3dZdy4cfLhhx/K7bffLgBk5syZyrKGDRsm/v7+UlxcbLWOzz//XPnsiYiYTCbp06ePeHl5ydixY+WTTz6RUaNGiZubmzz44IOVtrVdu3YSHBws06ZNk9mzZ8vOnTslIyNDIiIiJDIyUl5//XWZO3euPPDAAwJAPvjgA+X1Fy9elDZt2oiHh4dMnDhRZs6cKTExMdK5c+dKbV2V6q5n8+bNotFoZNy4ccq0xx9/XDw9PeXgwYPKtP/+979y++23y+uvvy7z58+XMWPGiKenp/To0UPMZrNSriH3qao+YyNGjBA3Nzd55plnZN68eTJp0iTx9vaW7t27S0lJiYiIZGZmSkBAgLRp00beffdd+fTTT+Xll1+Wdu3aXbNNRUT69+8vjz32mLz77rsyd+5cefTRRwWA/Pvf/1bKFBcXS4sWLSQ8PFzefPNNWbBggUybNk26d+8ux44du+byX3jhBenXr5+8/fbb8sknn8jw4cNFo9HII488cs3X/fXXX/LBBx8IABk0aJAsWrRIvvvuu6u2k8jl96oiANKlSxdp3LixvPHGGzJz5kxp2bKleHl5SXZ2tlLu9OnTEh4eruwL8+bNk1dffVXatWsnFy5ckCNHjsjo0aMFgLz00kvK90hGRoaIlH9v9+7dW1ledfdVy/f7TTfdJK1atZL//Oc/8s4770hQUJBEREQo7/HV2Pr7smKdunbtKu3bt5f3339fXnnlFdFqtdKrVy956aWX5JZbbpEPP/xQRo8eLSqVSoYOHWq1jOp8rkRE3n33XQEgP/zwg4iI5OfnS1RUlLRv316Kioquue1Vvd9vvvmmqFQqGThwoMyZM0emTZsmQUFB0rx5c7lw4YJSrnfv3hIeHi6RkZEyZswYmTNnjtx9990CQFavXq2Uy8/Pl5YtW4qnp6e8+OKLMnPmTOnRo4d06dLF6nvr559/lq5du0pQUJDSvpbPq+U74KabbpKYmBj54IMP5LXXXhMvLy/p0aPHNbfxSk4fio4cOSJubm4yevRoZb4tQtGzzz6rTCsrK5OIiAhRqVQyY8YMZfqFCxfE09NTEhMTlWmWN69JkyZiNBqV6cuWLRMAMmvWLBEpDwCtW7eW+Ph4qx+QixcvSosWLeTee++tVKdBgwZVq33Gjh0rAOS3335TpuXl5UmLFi2kefPmYjKZrLY/KSmpWstt1qyZJCQkWE2zZSgSEZk8ebLodDrJzc1VpmVlZYmbm5tVuapMmTJFAChfZBVZ2njmzJkCQL744gtlXklJicTGxoqPj4/ynq1du1YAyIoVK6yW069fP2nZsqXyfNGiRaJWq63aWkRk3rx5AkD++OMPq21Vq9Wyd+9eq7LDhw+Xxo0bW/3AiJQHET8/P+VH31L3ZcuWKWUKCgqkVatW1QpF1V2PSPn7oFarZdOmTbJ8+fJKP0QiYlXe4quvvhIAsmnTJmVaQ+1TIpU/Y7/99psAkMWLF1vVc82aNVbTv/vuO6uwWxNVtcNzzz0nXl5eyo/Szp07BYAsX77cJsufPn26qFQqOX78+DVfa/nue/fdd62m1zQUabVaOXz4sDLtr7/+EgDy0UcfKdOGDBkiarW6yja07H+Wz1JVn9UrQ1F191XLNjZq1EhycnKUsj/88EOV+/CV6uP70lKn4OBgq++yyZMnKyGztLRUmT5o0CDRarVWIaY6nyuR8v+Y3XbbbRIaGirZ2dmSlJQkbm5u1fosX/l+Hzt2TDQajbz11ltW5Xbv3i1ubm5W03v37i0A5H//+58yrbi4WMLCwqz+Y/vee+8JAPn++++VaYWFhRIdHV3ps5CQkFDl59LyHdCuXTur/6jOmjVLAMju3buvu60WTn36DABatmyJp556CvPnz8fZs2dtttwRI0Yof2s0GnTr1g0iguHDhyvT/f390bZtWxw9erTS64cMGQJfX1/l+SOPPILGjRtj9erVAIC0tDQcOnQITzzxBM6fP4/s7GxkZ2ejoKAA99xzDzZt2gSz2Wy1zJEjR1ar7qtXr0aPHj2sTrH5+Pjg2WefxbFjx7Bv377qNYIdDBkyBMXFxVZDL5cuXYqysjI8+eST13ztN998gy5duuChhx6qNM9yqHv16tUICwvDoEGDlHnu7u4YPXo08vPz8euvvwIA7r77bgQFBWHp0qVKuQsXLmDdunUYOHCgMm358uVo164doqOjlfcwOzsbd999N4DyU4IV9e7dG+3bt1eeiwi++eYb3H///RARq2XEx8fDYDBgx44dSt0bN26MRx55RHm9l5cXnn322Wu2S03XA5SfRunQoQMSExPxz3/+E71798bo0aOtlunp6an8XVRUhOzsbPTq1QsArJZlUd/7VFWWL18OPz8/3HvvvVbbHBMTAx8fH+X9sfQTWblyJUpLS6+6vKpUbIe8vDxkZ2fj9ttvx8WLF3HgwAEAgJ+fHwBg7dq1uHjxYq2XX1BQgOzsbNxyyy0QEezcubNGy6qtuLg4REVFKc87d+4MvV6vvE9msxnff/897r///ipHZNVm2Hd191WLgQMHIiAgQHl+++23A0CVn6Ur11Nf35ePPvqo8t4DQM+ePQEATz75JNzc3Kyml5SU4PTp08q06nyuAECtVmPhwoXIz89H3759MWfOHEyePLlWI+O+/fZbmM1mPPbYY1b7S1hYGFq3bl3p+8zHx8fqe1mr1aJHjx5Wbb5mzRo0adIEDzzwgDLNw8MDzzzzTI3rN3ToUKt+Z9V9jyty+lAEAK+88grKysqu27eoJpo2bWr13M/PDx4eHggKCqo0/cq+CQDQunVrq+cqlQqtWrVS+t4cOnQIQPlw2eDgYKvHggULUFxcXKmvQYsWLapV9+PHj6Nt27aVpluG0B8/frxay7GH6OhodO/eHYsXL1amLV68GL169UKrVq2u+dojR46gY8eO1yxz/PhxtG7dGmq19a5xZdu4ublhwIAB+OGHH5Q+O99++y1KS0utQtGhQ4ewd+/eSu9hmzZtAJT3iajoyvfw3LlzyM3Nxfz58ystw9KnybKM48ePo1WrVpV+YKp6r69Uk/UA5V9u//d//4f09HTk5eXhs88+q7TenJwcjBkzBqGhofD09ERwcLCyfVX1k6nvfaoqhw4dgsFgQEhISKXtzs/PV7a5d+/eGDBgAKZNm4agoCA8+OCD+Oyzz6rVX2vv3r146KGH4OfnB71ej+DgYOWHwtIOLVq0wPjx47FgwQIEBQUhPj4es2fPvm5/IgA4ceIEnn76aQQGBsLHxwfBwcHo3bu31fLr25XvHQAEBAQo79O5c+dgNBqvu//VRHX31avV0RKQqvosXbme+vq+rOozDwCRkZFVTq9Y1+p8riyioqLw2muvYdu2bejQoQNeffXVWtX30KFDEBG0bt260v6yf//+St9nERERlb4XKn4ugPL2i4qKqlTuet/nVante1yRUw/Jt2jZsiWefPJJzJ8/Hy+++GKl+Vf7X8qVHSEr0mg01ZoGlP8vvKYsR4HeffdddO3atcoyPj4+Vs8r/s/hRlGbtr2eIUOGYMyYMTh16hSKi4uxefNmfPzxx7VeXm09/vjj+OSTT/DTTz+hf//+WLZsGaKjo9GlSxeljNlsRqdOnfD+++9XuYwrv/yufA8tn4Mnn3wSiYmJVS6jc+fOddmMWq9n7dq1AMqPAh06dKhSoHvsscfw559/YsKECejatSt8fHxgNptx3333VTrKCdT/PlUVs9mMkJAQq5BdUXBwMAAoF4bbvHkzVqxYgbVr12LYsGF47733sHnz5kr7okVubi569+4NvV6P119/HVFRUfDw8MCOHTswadIkq3Z477338PTTT+OHH37Azz//jNGjR2P69OnYvHkzIiIiqly+yWTCvffei5ycHEyaNAnR0dHw9vbG6dOn8fTTT1fZztVR0/22vt8nW7gR63i1Ol2vrjX5XFlYLmlw5swZnD9/HmFhYTWur9lshkqlwk8//VRlHa/cDxq6zW2xPpcIRUD50aIvvvgC//nPfyrNs6TJ3Nxcq+n1ecTEciTIQkRw+PBh5YfHcihar9cjLi7Oputu1qwZDh48WGm65ZCrZUSKLdRH2z7++OMYP348vvrqKxQWFsLd3d3q6MzVREVFXfeCnc2aNcOuXbtgNput/gdaVdvccccdaNy4MZYuXYrbbrsN69evx8svv1xpnX/99RfuueeeWp0iCA4Ohq+vL0wm03U/B82aNcOePXsgIlbrquq9rst6gPKRZa+//jqGDh2KtLQ0jBgxArt377b6H21ycjKmTZuGKVOmKK+78nNvS9fbp6oSFRWFX375Bbfeemu1/lPRq1cv9OrVC2+99Ra+/PJLDB48GEuWLLE69VfRxo0bcf78eXz77be44447lOnp6elVlu/UqRM6deqEV155BX/++SduvfVWzJs3D2+++WaV5Xfv3o2///4bn3/+udUIo3Xr1l13W64lICCg0j4L1H6/DQ4Ohl6vv+7+V5N9pCb7al005PdlddX0czVv3jysW7cOb731FqZPn47nnnsOP/zwQ43XGxUVBRFBixYtlKPdddWsWTPs27ev0vdWVaPGGuLq2i5x+gwofzOffPJJfPLJJ8jIyLCap9frERQUhE2bNllNnzNnTr3V53//+x/y8vKU519//TXOnj2Lvn37AgBiYmIQFRWF//73v8jPz6/0esv1RWqjX79+2Lp1q9Xw9YKCAsyfPx/Nmze36tNSV5ZwV7FtTSYT5s+fX+tlBgUFoW/fvvjiiy+wePFi3HfffZVOsVRlwIAB+Ouvv6oc0m/5n0S/fv2QkZFh1VeorKwMH330EXx8fJTTEkD5ufpHHnkEK1aswKJFi1BWVlYpnD322GM4ffo0Pv3000rrLCwsvO71pDQaDQYMGIBvvvmmyh+Uip+Dfv364cyZM1b9rS5evFittq7JekpLS/H0008jPDwcs2bNwsKFC5GZmYlx48ZZLQ+o/D+0+rztwfX2qao89thjMJlMeOONNyrNKysrU4LBhQsXKm2L5QjutU6hVdUOJSUllb5bjEYjysrKrKZ16tQJarW6xssXEcyaNeuqr6mOqKgoGAwG7Nq1S5l29uzZKved6lCr1ejfvz9WrFiB7du3V5pvqb+3tzeAyv+JqkpN9tW6aMjvy+qq7ucKKA9KEyZMwIABA/DSSy/hv//9L3788Uf873//q/F6H374YWg0GkybNq3S/iAilS63UR3x8fE4ffo0fvzxR2VaUVFRld+Z3t7e9X5K2GWOFAHAyy+/jEWLFuHgwYPo0KGD1bwRI0ZgxowZGDFiBLp164ZNmzbh77//rre6BAYG4rbbbsPQoUORmZmJmTNnolWrVkrnMrVajQULFqBv377o0KEDhg4diiZNmuD06dPYsGED9Ho9VqxYUat1v/jii/jqq6/Qt29fjB49GoGBgfj888+Rnp6Ob775ptI5+rro0KEDevXqhcmTJyMnJweBgYFYsmRJpR+AmhoyZIjSobiqH7SqTJgwAV9//TUeffRRDBs2DDExMcjJycGPP/6IefPmoUuXLnj22WfxySef4Omnn0ZqaiqaN2+Or7/+Gn/88Qdmzpxp1ZEXKO+8+dFHH2Hq1Kno1KlTpVubPPXUU1i2bBlGjhyJDRs24NZbb4XJZMKBAwewbNkyrF279rodHmfMmIENGzagZ8+eeOaZZ9C+fXvk5ORgx44d+OWXX5TrlzzzzDP4+OOPMWTIEKSmpqJx48ZYtGgRvLy8qtU+1V3Pm2++ibS0NCQnJ8PX1xedO3fGlClT8Morr+CRRx5Bv379oNfrcccdd+Cdd95BaWkpmjRpgp9//vmq/5O1hevtU1Xp3bs3nnvuOUyfPh1paWno06cP3N3dcejQISxfvhyzZs3CI488gs8//xxz5szBQw89hKioKOTl5eHTTz+FXq9Hv379rrr8W265BQEBAUhMTMTo0aOhUqmwaNGiSj8o69evx6hRo/Doo4+iTZs2KCsrw6JFi5SwejXR0dGIiorCv//9b5w+fRp6vR7ffPNNjfpQVOXxxx/HpEmT8NBDD2H06NG4ePEi5s6dizZt2lTZSb463n77bfz888/o3bs3nn32WbRr1w5nz57F8uXL8fvvv8Pf3x9du3aFRqPBf/7zHxgMBuh0Otx9990ICQmptLya7qu11ZDfl9VV3c+ViGDYsGHw9PTE3LlzAQDPPfccvvnmG4wZMwZxcXEIDw+v9nqjoqLw5ptvYvLkyTh27Bj69+8PX19fpKen47vvvsOzzz5b46t5P/fcc/j4448xaNAgjBkzBo0bN8bixYvh4eEBwProUExMDJYuXYrx48eje/fu8PHxwf3331+j9V1XtcepOZCKQ/KvlJiYKACshuSLlA9vHD58uPj5+Ymvr6889thjkpWVddUh+efOnau0XG9v70rru3L4v2Xo4FdffSWTJ0+WkJAQ8fT0lISEhCqHz+7cuVMefvhhadSokeh0OmnWrJk89thjkpycfN06XcuRI0fkkUceEX9/f/Hw8JAePXrIypUrK5VDHYfkW9YVFxcnOp1OQkND5aWXXpJ169bVaki+RXFxsQQEBIifn58UFhZWq34iIufPn5dRo0ZJkyZNRKvVSkREhCQmJloNQ8/MzJShQ4dKUFCQaLVa6dSpU5WXaxC5fO0kAPLmm29WWaakpET+85//SIcOHUSn00lAQIDExMTItGnTxGAwWG3r1do6MzNTkpKSJDIyUtzd3SUsLEzuuecemT9/vlW548ePywMPPCBeXl4SFBQkY8aMUYaXX29IfnXWk5qaKm5ubvLCCy9Yva6srEy6d+8u4eHhyrVKTp06JQ899JD4+/uLn5+fPProo3LmzBm77lNXG2o+f/58iYmJEU9PT/H19ZVOnTrJxIkT5cyZMyIismPHDhk0aJA0bdpUdDqdhISEyD/+8Q/Zvn37ddv0jz/+kF69eomnp6eEh4fLxIkTlUs6WN6To0ePyrBhwyQqKko8PDwkMDBQ7rrrLvnll1+uu/x9+/ZJXFyc+Pj4SFBQkDzzzDPKkPirfW4trjYkX6T8ujAdO3YUrVYrbdu2lS+++OKqQ/Kr+tw2a9bM6tIJIuWfzyFDhkhwcLDodDpp2bKlJCUlWQ2j/vTTT6Vly5ai0Wis2ujKIfki1dtXr7WNV/t+uZKtvy+vVifLZ/nKSzNU9ZtWnc+VZUj6N998Y7W8EydOiF6vl379+l2znlW93yIi33zzjdx2223i7e0t3t7eEh0dLUlJSVbXKbtyP7Woah88evSoJCQkiKenpwQHB8u//vUv+eabbwSAbN68WSmXn58vTzzxhPj7+wsAZTlXa7drXW7nalQiN1BPOKJqKisrQ3h4OO6//378v//3/+xdHbKjjRs34q677sLy5cutLkdARI5r5syZGDduHE6dOlXppu71yWX6FJFz+f7773Hu3LlKl68nIiLHcuUtaYqKivDJJ5+gdevWDRqIABfrU0SOb8uWLdi1axfeeOMN3HTTTTbrTElERPbx8MMPo2nTpujatSsMBgO++OILHDhw4KqXyqhPDEXkUObOnYsvvvgCXbt2rfIGvkRE5Fji4+OxYMECLF68GCaTCe3bt8eSJUuqdakVW2OfIiIiIiKwTxERERERAIYiIiIiIgAu3qfIbDbjzJkz8PX1bZDLhxMREVHdiQjy8vIQHh5u0wtounQoOnPmTKUbchIREZFjOHny5FVvmFwbLh2KLJeBP3nyJPR6vZ1rQ9VWUABYLk1/5gxw6X5JRETkGoxGIyIjI212OxcLlw5FllNmer2eociRXLoZIgBAr2coIiJyUbbu+sKO1kRERERgKCIiIiICwFBEREREBIChiIiIiAgAQxERERERAIYiIiIiIgAMRUREREQAGIqIiIiIADAUEREREQFgKCIiIiICwFBEREREBIChiIiIiAiAi98QloiqR0RQZhaUmQQlJjNKLz3MApjNArNI+d8ikEt/my5Nl0vTTeYKDxGYzYCgvKyIQABALk0zA4LL0+XScsr/tn4dKk674nWosMxLRZVllP9tPd2yMLn85+V1WL3u8nRULIuK9al6esU2vV5ZqzVUqMeVy6ouQc1fVLv11OI1tXkRardNDfSSSp+R6r2mFuup+Usa7PMDAOPvbQNfD/davbahMRQR3eAsgaSo1ITiMjOKSk0oKjWjuOzyv8WlZqv5VytXVGpG0aXy5c8rv6akzIyyS+GlzCQoM5eHHyKi2nj+ziiGIiK6TERQUGLChYIS5BSUIOdiifL3hYslyCkohaGwBIbCUhgLy5BfXP4oKC5DUanphgwl7hoV1CrLA1CrK/ytUkGlUkGjhlJGpQI0alX5o8I0lUoFFXDp70uvRfkEFQD1lWWgUspW/FutUqH8ZRXLotK0S1OUvytOv7RmZZnKNKuyqipeZ70ey0Sr5SllrKdf/reaZSvUoTZq89IKW1XP66mdWq2rFi+qVf2crL1rsx4vreNEDcepKdENqqTMjHP5xcgwFCHTWHT53wp/ZxqLUVhqssn6dG5q6NzU8HDXQOeuhoebpvzvS9M83NXQuZXP07mVP7ea76aGrkI5j0vLsJTXuanhplHD7VKAufyvGhqNClqNGu4aVZ1+mImIbkQMRUTVVGYy42h2AfadMWLvGQP2nTXicFY+Mo3F1V6Gh7sagV5aBHhrEeitRYDX5X/9vdzh5+kOXw83+Hq4w0fnBm+d5lKQKQ8tWo0aajXDCBFRfWAoIqpCQXEZDmQYse+MEfvOGrH3jBEHM/JQXGausry7RoUQXw+E6nUI8/NAqN4DYXoP5e9QvQdCfHXw1nGXIyK6UfEbmlyeiOBodgE2Hz2PXScN2HY8B+nZBVWOzvDWahDdWI8O4Xq0b6xHmzBfNAv0QoCXlkdwiIgcHEMRuaRMYxH+OJyNPw6fx59HsnHWUFSpTIivDh3C9egQ7of2l0JQ00Avhh8iIifFUEQuwWwWpJ3Kxbp9mUjen4m/M/Ot5ms1asQ0C8BNTf1xc9MAdG3qjyAfnZ1qS0RE9sBQRE7rYkkZ/jx8HskHMvHL/iycy7vcIVqlAjo18cMtUUG4tVUjdGsWCE+txo61JSIie2MoIqciIkg5ch4L/zyGDQezUGq63DHIR+eGO9sG4972oejdJhj+Xlo71pSIiG40Nbr32WuvvVZ+cbIKj+joaGV+UVERkpKS0KhRI/j4+GDAgAHIzMy0WsaJEyeQkJAALy8vhISEYMKECSgrK7Mqs3HjRtx8883Q6XRo1aoVFi5cWKkus2fPRvPmzeHh4YGePXti69atNdkUcjKFJSYs3nIc8TM34YkFW/DzvkyUmgQRAZ4YEtsM/xvWAztevRcfP3EzHuzahIGIiIgqqfGRog4dOuCXX365vAC3y4sYN24cVq1aheXLl8PPzw+jRo3Cww8/jD/++AMAYDKZkJCQgLCwMPz55584e/YshgwZAnd3d7z99tsAgPT0dCQkJGDkyJFYvHgxkpOTMWLECDRu3Bjx8fEAgKVLl2L8+PGYN28eevbsiZkzZyI+Ph4HDx5ESEhInRqEHEumsQj/t+kElmw9CUNhKQDAS6vBgJsj8GSvZmgT6sOLDBIRUbWopAZ3rHvttdfw/fffIy0trdI8g8GA4OBgfPnll3jkkUcAAAcOHEC7du2QkpKCXr164aeffsI//vEPnDlzBqGhoQCAefPmYdKkSTh37hy0Wi0mTZqEVatWYc+ePcqyH3/8ceTm5mLNmjUAgJ49e6J79+74+OOPAQBmsxmRkZF44YUX8OKLL161/sXFxSguvtyvxGg0IjIyEgaDAXq9vrrNQPZWUAD4+AAAuk74Frnq8qM+TQO9kHhLczzaLQJ6B7nPDhER1ZzRaISfn5/Nf79rdPoMAA4dOoTw8HC0bNkSgwcPxokTJwAAqampKC0tRVxcnFI2OjoaTZs2RUpKCgAgJSUFnTp1UgIRAMTHx8NoNGLv3r1KmYrLsJSxLKOkpASpqalWZdRqNeLi4pQyVzN9+nT4+fkpj8jIyJpuPtlZSZkZ8zcdUZ4Xl5kR0ywAnw7phg3/vhPDb2vBQERERLVSo1DUs2dPLFy4EGvWrMHcuXORnp6O22+/HXl5ecjIyIBWq4W/v7/Va0JDQ5GRkQEAyMjIsApElvmWedcqYzQaUVhYiOzsbJhMpirLWJZxNZMnT4bBYFAeJ0+erMnmk51tO5aDf3z0Gz5Yd0iZNu+pm/H1yFjc2z4UGl4/iIiI6qBGfYr69u2r/N25c2f07NkTzZo1w7Jly+Dp6WnzytmaTqeDTsdrzzgaw8VSvL16P5ZuLw+xTbwvHwnq3SakdrdtJiIiukKNT59V5O/vjzZt2uDw4cMICwtDSUkJcnNzrcpkZmYiLCwMABAWFlZpNJrl+fXK6PV6eHp6IigoCBqNpsoylmWQ89hz2oD7P/5dCUSDekRi1ejb7VwrIiJyRnUKRfn5+Thy5AgaN26MmJgYuLu7Izk5WZl/8OBBnDhxArGxsQCA2NhY7N69G1lZWUqZdevWQa/Xo3379kqZisuwlLEsQ6vVIiYmxqqM2WxGcnKyUoacw8I/0vHQnD9wIuciIgM98fXIWEx/uDOH0xMRUf2QGvjXv/4lGzdulPT0dPnjjz8kLi5OgoKCJCsrS0RERo4cKU2bNpX169fL9u3bJTY2VmJjY5XXl5WVSceOHaVPnz6SlpYma9askeDgYJk8ebJS5ujRo+Ll5SUTJkyQ/fv3y+zZs0Wj0ciaNWuUMkuWLBGdTicLFy6Uffv2ybPPPiv+/v6SkZFRk80Rg8EgAMRgMNTodVT/Pkr+W5pNWinNJq2UZz7fJhcKii/PzM8XAcof+fn2qyQREdlFff1+16hP0alTpzBo0CCcP38ewcHBuO2227B582YEBwcDAD744AOo1WoMGDAAxcXFiI+Px5w5c5TXazQarFy5Es8//zxiY2Ph7e2NxMREvP7660qZFi1aYNWqVRg3bhxmzZqFiIgILFiwQLlGEQAMHDgQ586dw5QpU5CRkYGuXbtizZo1lTpfk2OaveEw/vvz3wCACfFt8c87o3itISIiqnc1uk6Rs6mv6xxQ7X266SjeWr0fQHkgSrqrVeVCFa5ThPx8wNu7AWtIRET2dsNcp4iovqzefVYJRP+6t03VgYiIiKieMBTRDWHz0fMYuzQNADD01uZ44Z7W9q0QERG5HIYisrv07AI887/tKCkzo0/7ULyS0N7eVSIiIhfEUER2VVRqQtLiHcgrKkNMswB8OOgmXpmaiIjsgqGI7Gr66v3Yd9aIAC93zH7iZni4a+xdJSIiclEMRWQ3P+0+i89TjgMA3h/YFWF+HnauERERuTKGIrKL07mFmPjNLgDAc71b4q62IXauERERuTqGIrKL137ci7yiMtzU1B//7tPW3tUhIiJiKKKG98u+TKzblwk3tQr/GdAZ7hp+DImIyP74a0QN6mJJGab+uBcAMOL2lmgT6mvnGhEREZVjKKIGNXvDYZzOLUQTf0+MvodXrCYiohsHQxE1mDO5hVjwWzoA4NV/tIeXtkb3IyYiIqpXDEXUYGb+8jeKy8zo0TwQ8R1C7V0dIiIiKwxF1CD+zszD16mnAAAv9ouGSsWrVhMR0Y2FoYgaxH9+OgCzAH07huHmpgH2rg4REVElDEVU77am5yD5QBY0ahX+Hc9rEhER0Y2JoYjq3YfJhwAAA7tHIirYx861ISIiqhpDEdWrv07m4vfD2XBTq/DPO6PsXR0iIqKrYiiiejVn42EAwINdmyAiwMvOtSEiIro6hiKqNydzLuLnfZkAgJG9W9q5NkRERNfGUET15n8pxyAC3NEmGK15Ow8iIrrBMRRRvSgqNWHZ9vLrEj19SzM714aIiOj6GIqoXqzdmwFDYSma+HvizjYh9q4OERHRdTEUUb1Ytv0kAOCRmAio1bx6NRER3fgYisjmTuZcxB+Hz0OlKg9FREREjoChiGxu+aV7nN0aFYTIQA7DJyIix8BQRDZlNgu+uRSKHuseaefaEBERVR9DEdlU6okLOJ1bCF+dG/q0D7V3dYiIiKqNoYhsasVfZwAAfTqEwcNdY+faEBERVR9DEdlMmcmM1bvPAgD+0aWxnWtDRERUMwxFZDNb0nOQnV8Cfy933NYqyN7VISIiqhGGIrIZy6mzvh0bw13DjxYRETkW/nKRTZSUmbFmbwYA4P7OPHVGRESOh6GIbOKPw9nIvViKIB8derZsZO/qEBER1RhDEdnEil3lp87+0bkxNLytBxEROSCGIqozk1mw4UAWAOC+jmF2rg0REVHtMBRRne06lYsLF0vhq3NDTLMAe1eHiIioVhiKqM42HjwHALitdRBHnRERkcPiLxjV2ca/y0PRnW2D7VwTIiKi2mMoojrJKSjBrlO5AIDebULsWxkiIqI6YCiiOtn09zmIANFhvgjz87B3dYiIiGqNoYjqZOPB8lFnd0XzKBERETk2hiKqNbNZsOlQNgDgzjbsT0RERI6NoYhqbc8ZA3IKSuCrc8PNHIpPREQOjqGIam3z0fMAgJ4tAzkUn4iIHB5/yajWNh/NAQD04r3OiIjICTAUUa2UmczYms5QREREzoOhiGpl7xkj8ovLoPdwQ7vGentXh4iIqM4YiqhWUi71J+rRohE0apWda0NERFR3DEVUK5ZO1rFRPHVGRETOgaGIaqzMZMY2pT9RoJ1rQ0REZBsMRVRju08bUFBigp+nO9qFsT8RERE5B4YiqjHLUPyeLQKhZn8iIiJyEnUKRTNmzIBKpcLYsWOVaUVFRUhKSkKjRo3g4+ODAQMGIDMz0+p1J06cQEJCAry8vBASEoIJEyagrKzMqszGjRtx8803Q6fToVWrVli4cGGl9c+ePRvNmzeHh4cHevbsia1bt9Zlc6iaLP2JOBSfiIicSa1D0bZt2/DJJ5+gc+fOVtPHjRuHFStWYPny5fj1119x5swZPPzww8p8k8mEhIQElJSU4M8//8Tnn3+OhQsXYsqUKUqZ9PR0JCQk4K677kJaWhrGjh2LESNGYO3atUqZpUuXYvz48Zg6dSp27NiBLl26ID4+HllZWbXdJKqGUpMZ246VHyliJ2siInIqUgt5eXnSunVrWbdunfTu3VvGjBkjIiK5ubni7u4uy5cvV8ru379fAEhKSoqIiKxevVrUarVkZGQoZebOnSt6vV6Ki4tFRGTixInSoUMHq3UOHDhQ4uPjlec9evSQpKQk5bnJZJLw8HCZPn16tbfDYDAIADEYDNXfeBeXejxHmk1aKV2mrRWTyWyfSuTniwDlj/x8+9SBiIjspr5+v2t1pCgpKQkJCQmIi4uzmp6amorS0lKr6dHR0WjatClSUlIAACkpKejUqRNCQ0OVMvHx8TAajdi7d69S5splx8fHK8soKSlBamqqVRm1Wo24uDilTFWKi4thNBqtHlQzKUcu3e+M/YmIiMjJuNX0BUuWLMGOHTuwbdu2SvMyMjKg1Wrh7+9vNT00NBQZGRlKmYqByDLfMu9aZYxGIwoLC3HhwgWYTKYqyxw4cOCqdZ8+fTqmTZtWvQ2lKinXJ2J/IiIicjI1OlJ08uRJjBkzBosXL4aHh0d91aneTJ48GQaDQXmcPHnS3lVyKKUmM7YfuwAA6MX+RERE5GRqFIpSU1ORlZWFm2++GW5ubnBzc8Ovv/6KDz/8EG5ubggNDUVJSQlyc3OtXpeZmYmwsDAAQFhYWKXRaJbn1yuj1+vh6emJoKAgaDSaKstYllEVnU4HvV5v9aDq23UqF4WlJgR6a9EmxNfe1SEiIrKpGoWie+65B7t370ZaWpry6NatGwYPHqz87e7ujuTkZOU1Bw8exIkTJxAbGwsAiI2Nxe7du61Gia1btw56vR7t27dXylRchqWMZRlarRYxMTFWZcxmM5KTk5UyZHu8PhERETmzGvUp8vX1RceOHa2meXt7o1GjRsr04cOHY/z48QgMDIRer8cLL7yA2NhY9OrVCwDQp08ftG/fHk899RTeeecdZGRk4JVXXkFSUhJ0Oh0AYOTIkfj4448xceJEDBs2DOvXr8eyZcuwatUqZb3jx49HYmIiunXrhh49emDmzJkoKCjA0KFD69QgdHWWofg9WvDWHkRE5Hxq3NH6ej744AOo1WoMGDAAxcXFiI+Px5w5c5T5Go0GK1euxPPPP4/Y2Fh4e3sjMTERr7/+ulKmRYsWWLVqFcaNG4dZs2YhIiICCxYsQHx8vFJm4MCBOHfuHKZMmYKMjAx07doVa9asqdT5mmxDRLD7lAEA0DXS376VISIiqgcqERF7V8JejEYj/Pz8YDAY2L/oOk7nFuLWGevhplZhz7R4eLhr7FeZggLAx6f87/x8wNvbfnUhIqIGV1+/37z3GVXL7lO5AIC2Yb72DURERET1hKGIquWvS6fOOkf42bkmRERE9YOhiKrF0p+oUxN/+1aEiIionjAU0XWJCHZdOn3GI0VEROSsGIrouk7kXISxqAxaNzXahPKijURE5JwYiui6LP2J2jXWQ+vGjwwRETkn/sLRdVlGnnVuwlNnRETkvBiK6Lp2ceQZERG5AIYiuiazWbDntCUU+du3MkRERPWIoYiu6Wh2PgpKTPB01yAqmFeOJiIi58VQRNdkOXXWIVwPNw0/LkRE5Lz4K0fXdLk/kb99K0JERFTPGIromnafZidrIiJyDQxFdFVlJjP2nrl0ew+GIiIicnIMRXRVh7LyUVRqhq/ODS0asZM1ERE5N4YiuirLTWA7NvGDWq2yc22IiIjqF0MRXdWu07kA2J+IiIhcA0MRXZXlSBH7ExERkStgKKIqlZSZsf9sHgCgC4fjExGRC2AooiodzMhDickMfy93RAR42rs6RERE9Y6hiKpk6U/UqYkfVCp2siYiIufHUERV2n2KF20kIiLXwlBEVfqLt/cgIiIXw1BElRSVmvB3Znknax4pIiIiV8FQRJXsP2uEySwI8tEiTO9h7+oQERE1CIYiqsQyFL9dYz07WRMRkctgKKJK9p81AgDaN9bbuSZEREQNh6GIKrGEonYMRURE5EIYisiK2Sw4kHH59BkREZGrYCgiKycvXER+cRm0GjVaBnvbuzpEREQNhqGIrFhOnbUO9YG7hh8PIiJyHfzVIyv7zvLUGRERuSaGIrJygJ2siYjIRTEUkZWDl65k3S7M1841ISIialgMRaQoKC7D8fMXAQBtGYqIiMjFMBSRwnKUKNhXh0Y+OjvXhoiIqGExFJHi4KXrE0XzKBEREbkghiJSWDpZMxQREZErYigixQHlSBFHnhERkethKCIAgMjl23tEN+aRIiIicj0MRQQAyDAWwVBYCo1ahVYhPvauDhERUYNjKCIAl0+dtQzyhs5NY+faEBERNTyGIgIAHDhrOXXG/kREROSaGIoIAHAggyPPiIjItTEUEQBeo4iIiIihiFBSZsbhrHwAvL0HERG5LoYiwpFz+SgzC3x1bmji72nv6hAREdkFQxFdPnXW2BcqlcrOtSEiIrIPhiLC/kudrHnqjIiIXBlDEV0ejs/bexARkQtjKCLl9Fk73t6DiIhcGEORi7tQUIIMYxEAoE0oQxEREbkuhiIXZ7m9R0SAJ3w93O1cGyIiIvthKHJxB5UrWbM/ERERubYahaK5c+eic+fO0Ov10Ov1iI2NxU8//aTMLyoqQlJSEho1agQfHx8MGDAAmZmZVss4ceIEEhIS4OXlhZCQEEyYMAFlZWVWZTZu3Iibb74ZOp0OrVq1wsKFCyvVZfbs2WjevDk8PDzQs2dPbN26tSabQpcc4JWsiYiIANQwFEVERGDGjBlITU3F9u3bcffdd+PBBx/E3r17AQDjxo3DihUrsHz5cvz66684c+YMHn74YeX1JpMJCQkJKCkpwZ9//onPP/8cCxcuxJQpU5Qy6enpSEhIwF133YW0tDSMHTsWI0aMwNq1a5UyS5cuxfjx4zF16lTs2LEDXbp0QXx8PLKysuraHi5nf4VrFBEREbk0qaOAgABZsGCB5Obmiru7uyxfvlyZt3//fgEgKSkpIiKyevVqUavVkpGRoZSZO3eu6PV6KS4uFhGRiRMnSocOHazWMXDgQImPj1ee9+jRQ5KSkpTnJpNJwsPDZfr06TWqu8FgEABiMBhq9DpnYTKZJfqVn6TZpJVyKNNo7+pUX36+CFD+yM+3d22IiKiB1dfvd637FJlMJixZsgQFBQWIjY1FamoqSktLERcXp5SJjo5G06ZNkZKSAgBISUlBp06dEBoaqpSJj4+H0WhUjjalpKRYLcNSxrKMkpISpKamWpVRq9WIi4tTylxNcXExjEaj1cOVnci5iMJSE7RuajRv5G3v6hAREdlVjUPR7t274ePjA51Oh5EjR+K7775D+/btkZGRAa1WC39/f6vyoaGhyMjIAABkZGRYBSLLfMu8a5UxGo0oLCxEdnY2TCZTlWUsy7ia6dOnw8/PT3lERkbWdPOdyoFLnazbhPrATcM+90RE5Npq/EvYtm1bpKWlYcuWLXj++eeRmJiIffv21UfdbG7y5MkwGAzK4+TJk/aukl1ZOlm3DeXIMyIiIreavkCr1aJVq1YAgJiYGGzbtg2zZs3CwIEDUVJSgtzcXKujRZmZmQgLCwMAhIWFVRolZhmdVrHMlSPWMjMzodfr4enpCY1GA41GU2UZyzKuRqfTQafT1XSTnZbl9h68kjUREZENrlNkNptRXFyMmJgYuLu7Izk5WZl38OBBnDhxArGxsQCA2NhY7N6922qU2Lp166DX69G+fXulTMVlWMpYlqHVahETE2NVxmw2Izk5WSlD1XMw89KRIg7HJyIiqtmRosmTJ6Nv375o2rQp8vLy8OWXX2Ljxo1Yu3Yt/Pz8MHz4cIwfPx6BgYHQ6/V44YUXEBsbi169egEA+vTpg/bt2+Opp57CO++8g4yMDLzyyitISkpSjuCMHDkSH3/8MSZOnIhhw4Zh/fr1WLZsGVatWqXUY/z48UhMTES3bt3Qo0cPzJw5EwUFBRg6dKgNm8a5FZaYcOx8AQCGIiIiIqCGoSgrKwtDhgzB2bNn4efnh86dO2Pt2rW49957AQAffPAB1Go1BgwYgOLiYsTHx2POnDnK6zUaDVauXInnn38esbGx8Pb2RmJiIl5//XWlTIsWLbBq1SqMGzcOs2bNQkREBBYsWID4+HilzMCBA3Hu3DlMmTIFGRkZ6Nq1K9asWVOp8zVd3eGsfIgAgd5aBPvwlCIREZFKRMTelbAXo9EIPz8/GAwG6PWu1dl4+faTmPD1LvRqGYglzzrYaceCAsDHp/zv/HzAm5cTICJyJfX1+81x2C7qoHJ7D9cKg0RERFfDUOSi2MmaiIjIGkORi7IcKWoTylBEREQEMBS5pAsFJcjKKwbAI0VEREQWDEUuyHLqLCLAEz66Gl+/k4iIyCkxFLmgy52seZSIiIjIgqHIBR3KKg9FrUIYioiIiCwYilzQkazyK1m3CvGxc02IiIhuHAxFLujwuXwADEVEREQVMRS5GENhKc5dGnkWFcwrQRMREVkwFLmYw1nlR4nC9B7w9XC3c22IiIhuHAxFLuZIFk+dERERVYWhyMWwPxEREVHVGIpcjOX0WRRDERERkRWGIhdz5NKRInayJiIissZQ5EKKSk04mXMRAE+fERERXYmhyIWkZxfALIDeww3BPjp7V4eIiOiGwlDkQg5XGHmmUqnsXBsiIqIbC0ORCznM4fhERERXxVDkQjgcn4iI6OoYilyI5cKNUcEMRURERFdiKHIRJrPgaHYBAB4pIiIiqgpDkYs4deEiSsrM0LqpERHgZe/qEBER3XAYilyEpZN1yyBvaNQceUZERHQlhiIXwZFnRERE18ZQ5CIYioiIiK6NochFcDg+ERHRtTEUuQARUY4UcTg+ERFR1RiKXMC5vGLkFZVBrQJaBHnbuzpEREQ3JIYiF2A5dRYZ6AUPd42da0NERHRjYihyAZYrWbfiqTMiIqKrYihyARx5RkREdH0MRS4g/fxFAOxPREREdC0MRS7g+Pnye541ZygiIiK6KoYiJ1dqMuPUhUIAQPNGDEVERERXw1Dk5E5fKITJLPBwVyPEV2fv6hAREd2wGIqc3LFLp86aBXpDzRvBEhERXRVDkZM7fqmTdbNGXnauCRER0Y2NocjJHWMnayIiomphKHJyJ3ikiIiIqFoYipyccqSII8+IiIiuiaHIiZnMgpM55cPxeaSIiIjo2hiKnNhZQyFKTGZoNWo09vO0d3WIiIhuaAxFTswy8iwy0BMaDscnIiK6JoYiJ3YixxKKeOqMiIjoehiKnNjZ3PL+RE38eeqMiIjoehiKnNjp3CIAQDhDERER0XUxFDmxs4byI0Xh/h52rgkREdGNj6HIiZ25dPqMI8+IiIiuj6HISYkIzhrKT5+xTxEREdH1MRQ5qZyCEhSXmaFSAaF6nj4jIiK6HoYiJ3XmUifrYB8dtG58m4mIiK6Hv5ZO6sylTtaNeeqMiIioWmoUiqZPn47u3bvD19cXISEh6N+/Pw4ePGhVpqioCElJSWjUqBF8fHwwYMAAZGZmWpU5ceIEEhIS4OXlhZCQEEyYMAFlZWVWZTZu3Iibb74ZOp0OrVq1wsKFCyvVZ/bs2WjevDk8PDzQs2dPbN26tSab49TOKNco4qkzIiKi6qhRKPr111+RlJSEzZs3Y926dSgtLUWfPn1QUFCglBk3bhxWrFiB5cuX49dff8WZM2fw8MMPK/NNJhMSEhJQUlKCP//8E59//jkWLlyIKVOmKGXS09ORkJCAu+66C2lpaRg7dixGjBiBtWvXKmWWLl2K8ePHY+rUqdixYwe6dOmC+Ph4ZGVl1aU9nIalkzVHnhEREVWT1EFWVpYAkF9//VVERHJzc8Xd3V2WL1+ulNm/f78AkJSUFBERWb16tajVasnIyFDKzJ07V/R6vRQXF4uIyMSJE6VDhw5W6xo4cKDEx8crz3v06CFJSUnKc5PJJOHh4TJ9+vRq199gMAgAMRgMNdhqx/DPxanSbNJKWfDbUXtXxfby80WA8kd+vr1rQ0REDay+fr/r1KfIYDAAAAIDAwEAqampKC0tRVxcnFImOjoaTZs2RUpKCgAgJSUFnTp1QmhoqFImPj4eRqMRe/fuVcpUXIaljGUZJSUlSE1NtSqjVqsRFxenlKlKcXExjEaj1cNZWW7xEe7H02dERETVUetQZDabMXbsWNx6663o2LEjACAjIwNarRb+/v5WZUNDQ5GRkaGUqRiILPMt865Vxmg0orCwENnZ2TCZTFWWsSyjKtOnT4efn5/yiIyMrPmGO4gzvMUHERFRjdQ6FCUlJWHPnj1YsmSJLetTryZPngyDwaA8Tp48ae8q1YtSkxlZeZf6FLGjNRERUbW41eZFo0aNwsqVK7Fp0yZEREQo08PCwlBSUoLc3Fyro0WZmZkICwtTylw5SswyOq1imStHrGVmZkKv18PT0xMajQYajabKMpZlVEWn00Gn09V8gx1MVl4xzAK4a1QI8nb+7SUiIrKFGh0pEhGMGjUK3333HdavX48WLVpYzY+JiYG7uzuSk5OVaQcPHsSJEycQGxsLAIiNjcXu3butRomtW7cOer0e7du3V8pUXIaljGUZWq0WMTExVmXMZjOSk5OVMq4s49I1ikL1HlCrVXauDRERkWOo0ZGipKQkfPnll/jhhx/g6+ur9N/x8/ODp6cn/Pz8MHz4cIwfPx6BgYHQ6/V44YUXEBsbi169egEA+vTpg/bt2+Opp57CO++8g4yMDLzyyitISkpSjuKMHDkSH3/8MSZOnIhhw4Zh/fr1WLZsGVatWqXUZfz48UhMTES3bt3Qo0cPzJw5EwUFBRg6dKit2sZhXR6Oz1NnRERE1VaToWoAqnx89tlnSpnCwkL55z//KQEBAeLl5SUPPfSQnD171mo5x44dk759+4qnp6cEBQXJv/71LyktLbUqs2HDBunatatotVpp2bKl1TosPvroI2natKlotVrp0aOHbN68uSab47RD8j/ddESaTVopo77cYe+q1A8OyScicmn19futEhGxXySzL6PRCD8/PxgMBuj1entXx2beXLkPC35PxzO3t8DLCe3tXR3bKygAfHzK/87PB7y97VsfIiJqUPX1+817nzmhs8by02dhvJo1ERFRtTEUOaEM9ikiIiKqMYYiJ2QJRWEMRURERNXGUORkzGZBpuX0mZ6hiIiIqLoYipxMdkExyswCtQoI9uWFG4mIiKqLocjJWE6dBfvq4K7h20tERFRd/NV0MmcNHHlGRERUGwxFTsbSn6gx+xMRERHVCEORk8kyFgMAQvTsT0RERFQTDEVO5lxeeSgK8mEoIiIiqgmGIieTnc9QREREVBsMRU7mcijS2rkmREREjoWhyMlk55cAAIJ4jSIiIqIaYShyIiKCc5eOFAXz9BkREVGNMBQ5kbziMpSUmQGwTxEREVFNMRQ5kexLI898dG7w1GrsXBsiIiLHwlDkRJT+ROxkTUREVGMMRU6E1ygiIiKqPYYiJ8JrFBEREdUeQ5ETUUKRL0+fERER1RRDkRPhkSIiIqLaYyhyIufyLB2tGYqIiIhqiqHIifBIERERUe0xFDkRSygK5i0+iIiIaoyhyEmIyOVQxCNFRERENcZQ5CQKSkwoKr10iw+OPiMiIqoxhiInYblwo5dWAy+tm51rQ0RE5HgYipwEO1kTERHVDUORk8hWbvHBU2dERES1wVDkJHikiIiIqG4YipzEufxLF27kcHwiIqJaYShyEhyOT0REVDcMRU5C6VPEI0VERES1wlDkJC4fKWJHayIiotpgKHIS2fm8GSwREVFdMBQ5iXN5HH1GRERUFwxFTqCguAyFpSYA7FNERERUWwxFTsDSn8jDXQ1vrcbOtSEiInJMDEVOoOKFG1UqlZ1rQ0RE5JgYipzAubzyTtbBPHVGRERUawxFToC3+CAiIqo7hiInwFBERERUdwxFToAXbiQiIqo7hiIncI63+CAiIqozhiInwKtZExER1R1DkRNgnyIiIqK6YyhyAtnKLT7Yp4iIiKi2GIocXGGJCQUlvMUHERFRXTEUOTjLqTOdmxq+Ojc714aIiMhxMRQ5uHO8xQcREZFNMBQ5uGwOxyciIrIJhiIHZxmOzws3EhER1Q1DkYNTLtzI4fhERER1wlDk4HiNIiIiItuocSjatGkT7r//foSHh0OlUuH777+3mi8imDJlCho3bgxPT0/ExcXh0KFDVmVycnIwePBg6PV6+Pv7Y/jw4cjPz7cqs2vXLtx+++3w8PBAZGQk3nnnnUp1Wb58OaKjo+Hh4YFOnTph9erVNd0ch5eVVwQACNEzFBEREdVFjUNRQUEBunTpgtmzZ1c5/5133sGHH36IefPmYcuWLfD29kZ8fDyKioqUMoMHD8bevXuxbt06rFy5Eps2bcKzzz6rzDcajejTpw+aNWuG1NRUvPvuu3jttdcwf/58pcyff/6JQYMGYfjw4di5cyf69++P/v37Y8+ePTXdJIeWaSw/UhTi62HnmhARETk4qQMA8t133ynPzWazhIWFybvvvqtMy83NFZ1OJ1999ZWIiOzbt08AyLZt25QyP/30k6hUKjl9+rSIiMyZM0cCAgKkuLhYKTNp0iRp27at8vyxxx6ThIQEq/r07NlTnnvuuWrX32AwCAAxGAzVfs2NJvbtX6TZpJWy88QFe1el4eTniwDlj/x8e9eGiIgaWH39ftu0T1F6ejoyMjIQFxenTPPz80PPnj2RkpICAEhJSYG/vz+6deumlImLi4NarcaWLVuUMnfccQe02ssjquLj43Hw4EFcuHBBKVNxPZYylvVUpbi4GEaj0erhyMxmQdaljtahPH1GRERUJzYNRRkZGQCA0NBQq+mhoaHKvIyMDISEhFjNd3NzQ2BgoFWZqpZRcR1XK2OZX5Xp06fDz89PeURGRtZ0E28oORdLUGYWqFTsaE1ERFRXLjX6bPLkyTAYDMrj5MmT9q5SnWQay/tpNfLWwV3jUm8lERGRzdn0lzQsLAwAkJmZaTU9MzNTmRcWFoasrCyr+WVlZcjJybEqU9UyKq7jamUs86ui0+mg1+utHo4sy8hTZ0RERLZi01DUokULhIWFITk5WZlmNBqxZcsWxMbGAgBiY2ORm5uL1NRUpcz69ethNpvRs2dPpcymTZtQWlqqlFm3bh3atm2LgIAApUzF9VjKWNbjCixHikL1HHlGRERUVzUORfn5+UhLS0NaWhqA8s7VaWlpOHHiBFQqFcaOHYs333wTP/74I3bv3o0hQ4YgPDwc/fv3BwC0a9cO9913H5555hls3boVf/zxB0aNGoXHH38c4eHhAIAnnngCWq0Ww4cPx969e7F06VLMmjUL48ePV+oxZswYrFmzBu+99x4OHDiA1157Ddu3b8eoUaPq3ioOIpNHioiIiGynpsPVNmzYIAAqPRITE0WkfFj+q6++KqGhoaLT6eSee+6RgwcPWi3j/PnzMmjQIPHx8RG9Xi9Dhw6VvLw8qzJ//fWX3HbbbaLT6aRJkyYyY8aMSnVZtmyZtGnTRrRarXTo0EFWrVpVo21x9CH5k7/dJc0mrZT3fz54/cLOhEPyiYhcWn39fqtEROyYyezKaDTCz88PBoPBIfsXjfh8G37Zn4W3H+qEJ3o2tXd1Gk5BAeDjU/53fj7g7W3f+hARUYOqr99vDllyYDx9RkREZDsMRQ6MHa2JiIhsh6HIQZWZzMjOv3TfMx4pIiIiqjOGIgd1vqAEZgE0ahUaeTMUERER1RVDkYOynDoL9tFBo1bZuTZERESOj6HIQbGTNRERkW0xFDkoy5GiEHayJiIisgmGIgeVpYw845EiIiIiW2AoclAZllDkyyNFREREtsBQ5KAu9yliKCIiIrIFhiIHdblPEU+fERER2QJDkYPKyis/UhTmxyNFREREtsBQ5ICKy0zIKSgBAISwTxEREZFNMBQ5oDO55afOPN01CPByt3NtiIiInANDkQM6faEQANAkwBMqFa9mTUREZAsMRQ7o1IWLAICIAE8714SIiMh5MBQ5oNO5l44U+TMUERER2QpDkQM6den0WUSAl51rQkRE5DwYihxQxT5FREREZBsMRQ6IfYqIiIhsj6HIwZSazMp9zxiKiIiIbIehyMFkGIpgFkDrpkaQN2/xQUREZCsMRQ7mpOXUmb8n1Gpeo4iIiMhWGIocDDtZExER1Q+GIgdzeTg+QxEREZEtMRQ5GF64kYiIqH4wFDmYy8PxeeFGIiIiW2IocjCn2KeIiIioXjAUOZAykxkZBl6jiIiIqD4wFDmQs4YilJkF7hoVQnw97F0dIiIip8JQ5ECOnS8AADQN9IKG1ygiIiKyKYYiB3IsuzwUtQjytnNNiIiInA9DkQM5ylBERERUbxiKHIjlSFFzhiIiIiKbYyhyIOk8UkRERFRvGIocRKnJjJOXrlHEUERERGR7DEUO4tSFQpjMAk93DUI5HJ+IiMjmGIocRHp2PgCgWSMvqDkcn4iIyOYYihzE0XPl/YlaBvPUGRERUX1gKHIQR86VHymKCvaxc02IiIicE0ORg/g7szwUtQ71tXNNiIiInBNDkQMQEfydmQcAaB3CI0VERET1gaHIAWQai5FXVAaNWsU+RURERPWEocgBWI4SNWvkBZ2bxs61ISIick4MRQ7AEorahLA/ERERUX1hKHIAe88YAQDRjRmKiIiI6gtDkQNIO5kLAOga6W/XehARETkzhqIbXO7FEuVGsAxFRERE9Yeh6Ab31ykDAKB5Iy/4e2ntXBsiIiLnxVB0g0s7kQuAR4mIiIjqG0PRDe6vU7kAGIqIiIjqG0PRDaykzIztx3IAAF0YioiIiOoVQ9EN7I/D2TAWlSHYV4fOEf72rg4REZFTYyi6ga3YdQYA0K9jGDRqlZ1rQ0RE5NwcPhTNnj0bzZs3h4eHB3r27ImtW7fau0o2YSgsxdo9GQCAhM7hdq4NERGR83PoULR06VKMHz8eU6dOxY4dO9ClSxfEx8cjKyvL3lWrs083HUVBiQnRYb7o3jzA3tUhIiJyeioREXtXorZ69uyJ7t274+OPPwYAmM1mREZG4oUXXsCLL75YqXxxcTGKi4uV50ajEZGRkTAYDNDr9Tar1/s/H4SxqAwiAgEgAgjk0r/lz2F5XmGe+dLfFwpKsOHgOQDA7CduRkLnxjarm1MoKAB8fMr/zs8HvL3tWx8iImpQRqMRfn5+Nv/9drPZkhpYSUkJUlNTMXnyZGWaWq1GXFwcUlJSqnzN9OnTMW3atHqv21fbTuJcXvH1C17Hc71bol+nMBvUiIiIiK7HYUNRdnY2TCYTQkNDraaHhobiwIEDVb5m8uTJGD9+vPLccqTI1obe2hwXi01QqQAVAKhUUJX/AxVUynSVClCpyjtQq1WXp2vUKvRq2Qgdm/jZvG5ERERUNYcNRbWh0+mg0+nqfT3/vLNVva+DiIiIbMthO1oHBQVBo9EgMzPTanpmZibCwnjKiYiIiGrGYUORVqtFTEwMkpOTlWlmsxnJycmIjY21Y82IiIjIETn06bPx48cjMTER3bp1Q48ePTBz5kwUFBRg6NCh9q4aERERORiHDkUDBw7EuXPnMGXKFGRkZKBr165Ys2ZNpc7XRERERNfj0Ncpqqv6us4B1TNep4iIyKXV1++3w/YpIiIiIrIlhiIiIiIiMBQRERERAWAoIiIiIgLAUEREREQEgKGIiIiICABDEREREREAhiIiIiIiAA5+Reu6sly30mg02rkmVCMFBZf/NhoBk8l+dSEiogZn+d229fWnXToU5eXlAQAiIyPtXBOqtfBwe9eAiIjsJC8vD35+fjZbnkvf5sNsNuPMmTPw9fWFSqWy2XKNRiMiIyNx8uRJ3j7kErZJZWyTytgmVWO7VMY2qcyV2kREkJeXh/DwcKjVtusJ5NJHitRqNSIiIupt+Xq93uk/mDXFNqmMbVIZ26RqbJfK2CaVuUqb2PIIkQU7WhMRERGBoYiIiIgIAENRvdDpdJg6dSp0Op29q3LDYJtUxjapjG1SNbZLZWyTytgmdefSHa2JiIiILHikiIiIiAgMRUREREQAGIqIiIiIADAUEREREQFgKCIiIiICwFBUL2bPno3mzZvDw8MDPXv2xNatW+1dpXoxffp0dO/eHb6+vggJCUH//v1x8OBBqzJFRUVISkpCo0aN4OPjgwEDBiAzM9OqzIkTJ5CQkAAvLy+EhIRgwoQJKCsra8hNqTczZsyASqXC2LFjlWmu2CanT5/Gk08+iUaNGsHT0xOdOnXC9u3blfkigilTpqBx48bw9PREXFwcDh06ZLWMnJwcDB48GHq9Hv7+/hg+fDjy8/MbelNswmQy4dVXX0WLFi3g6emJqKgovPHGG1Y3t3SFNtm0aRPuv/9+hIeHQ6VS4fvvv7eab6s22LVrF26//XZ4eHggMjIS77zzTn1vWq1dq01KS0sxadIkdOrUCd7e3ggPD8eQIUNw5swZq2U4W5s0KCGbWrJkiWi1Wvm///s/2bt3rzzzzDPi7+8vmZmZ9q6azcXHx8tnn30me/bskbS0NOnXr580bdpU8vPzlTIjR46UyMhISU5Olu3bt0uvXr3klltuUeaXlZVJx44dJS4uTnbu3CmrV6+WoKAgmTx5sj02yaa2bt0qzZs3l86dO8uYMWOU6a7WJjk5OdKsWTN5+umnZcuWLXL06FFZu3atHD58WCkzY8YM8fPzk++//17++usveeCBB6RFixZSWFiolLnvvvukS5cusnnzZvntt9+kVatWMmjQIHtsUp299dZb0qhRI1m5cqWkp6fL8uXLxcfHR2bNmqWUcYU2Wb16tbz88svy7bffCgD57rvvrObbog0MBoOEhobK4MGDZc+ePfLVV1+Jp6enfPLJJw21mTVyrTbJzc2VuLg4Wbp0qRw4cEBSUlKkR48eEhMTY7UMZ2uThsRQZGM9evSQpKQk5bnJZJLw8HCZPn26HWvVMLKysgSA/PrrryJSvgO7u7vL8uXLlTL79+8XAJKSkiIi5V8AarVaMjIylDJz584VvV4vxcXFDbsBNpSXlyetW7eWdevWSe/evZVQ5IptMmnSJLntttuuOt9sNktYWJi8++67yrTc3FzR6XTy1VdfiYjIvn37BIBs27ZNKfPTTz+JSqWS06dP11/l60lCQoIMGzbMatrDDz8sgwcPFhHXbJMrA4Ct2mDOnDkSEBBgte9MmjRJ2rZtW89bVHdVBcUrbd26VQDI8ePHRcT526S+8fSZDZWUlCA1NRVxcXHKNLVajbi4OKSkpNixZg3DYDAAAAIDAwEAqampKC0ttWqP6OhoNG3aVGmPlJQUdOrUCaGhoUqZ+Ph4GI1G7N27twFrb1tJSUlISEiw2nbANdvkxx9/RLdu3fDoo48iJCQEN910Ez799FNlfnp6OjIyMqzaxM/PDz179rRqE39/f3Tr1k0pExcXB7VajS1btjTcxtjILbfcguTkZPz9998AgL/++gu///47+vbtC8A12+RKtmqDlJQU3HHHHdBqtUqZ+Ph4HDx4EBcuXGigrak/BoMBKpUK/v7+ANgmdeVm7wo4k+zsbJhMJqsfMwAIDQ3FgQMH7FSrhmE2mzF27Fjceuut6NixIwAgIyMDWq1W2VktQkNDkZGRoZSpqr0s8xzRkiVLsGPHDmzbtq3SPFdsk6NHj2Lu3LkYP348XnrpJWzbtg2jR4+GVqtFYmKisk1VbXPFNgkJCbGa7+bmhsDAQIdskxdffBFGoxHR0dHQaDQwmUx46623MHjwYABwyTa5kq3aICMjAy1atKi0DMu8gICAeql/QygqKsKkSZMwaNAg6PV6AGyTumIoIptISkrCnj178Pvvv9u7KnZ18uRJjBkzBuvWrYOHh4e9q3NDMJvN6NatG95++20AwE033YQ9e/Zg3rx5SExMtHPt7GPZsmVYvHgxvvzyS3To0AFpaWkYO3YswsPDXbZNqGZKS0vx2GOPQUQwd+5ce1fHafD0mQ0FBQVBo9FUGkmUmZmJsLAwO9Wq/o0aNQorV67Ehg0bEBERoUwPCwtDSUkJcnNzrcpXbI+wsLAq28syz9GkpqYiKysLN998M9zc3ODm5oZff/0VH374Idzc3BAaGupybdK4cWO0b9/ealq7du1w4sQJAJe36Vr7TVhYGLKysqzml5WVIScnxyHbZMKECXjxxRfx+OOPo1OnTnjqqacwbtw4TJ8+HYBrtsmVbNUGzrY/AZcD0fHjx7Fu3TrlKBHgum1iKwxFNqTVahETE4Pk5GRlmtlsRnJyMmJjY+1Ys/ohIhg1ahS+++47rF+/vtLh2JiYGLi7u1u1x8GDB3HixAmlPWJjY7F7926rndiyk1/5Q+oI7rnnHuzevRtpaWnKo1u3bhg8eLDyt6u1ya233lrpUg1///03mjVrBgBo0aIFwsLCrNrEaDRiy5YtVm2Sm5uL1NRUpcz69ethNpvRs2fPBtgK27p48SLUauuvX41GA7PZDMA12+RKtmqD2NhYbNq0CaWlpUqZdevWoW3btg55msgSiA4dOoRffvkFjRo1sprvim1iU/bu6e1slixZIjqdThYuXCj79u2TZ599Vvz9/a1GEjmL559/Xvz8/GTjxo1y9uxZ5XHx4kWlzMiRI6Vp06ayfv162b59u8TGxkpsbKwy3zL8vE+fPpKWliZr1qyR4OBghx1+XpWKo89EXK9Ntm7dKm5ubvLWW2/JoUOHZPHixeLl5SVffPGFUmbGjBni7+8vP/zwg+zatUsefPDBKode33TTTbJlyxb5/fffpXXr1g41/LyixMREadKkiTIk/9tvv5WgoCCZOHGiUsYV2iQvL0927twpO3fuFADy/vvvy86dO5WRVLZog9zcXAkNDZWnnnpK9uzZI0uWLBEvL68bdvj5tdqkpKREHnjgAYmIiJC0tDSr792KI8mcrU0aEkNRPfjoo4+kadOmotVqpUePHrJ582Z7V6leAKjy8dlnnyllCgsL5Z///KcEBASIl5eXPPTQQ3L27Fmr5Rw7dkz69u0rnp6eEhQUJP/617+ktLS0gbem/lwZilyxTVasWCEdO3YUnU4n0dHRMn/+fKv5ZrNZXn31VQkNDRWdTif33HOPHDx40KrM+fPnZdCgQeLj4yN6vV6GDh0qeXl5DbkZNmM0GmXMmDHStGlT8fDwkJYtW8rLL79s9cPmCm2yYcOGKr9DEhMTRcR2bfDXX3/JbbfdJjqdTpo0aSIzZsxoqE2ssWu1SXp6+lW/dzds2KAsw9napCGpRCpcQpWIiIjIRbFPEREREREYioiIiIgAMBQRERERAWAoIiIiIgLAUEREREQEgKGIiIiICABDEREREREAhiIiIiIiAAxFRERERAAYioiIiIgAMBQRERERAQD+P70xpn+pwq6HAAAAAElFTkSuQmCC",
+      "text/plain": [
+       "<Figure size 640x480 with 1 Axes>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "plt.plot([len([l for l in lens if l <= m]) for m in range(max(lens) + 1)])\n",
+    "plt.title(\"Number of fully covered examples as a function of max length\")\n",
+    "plt.axvline(x=256, color=\"red\")"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Percentage of tokens left out:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "<matplotlib.lines.Line2D at 0x7f6eef392020>"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhYAAAGzCAYAAABzfl4TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABHN0lEQVR4nO3deVxU9f4/8NfMwAz7sIMoAqJprplb5q6UmllWWnnN1G7eb2WlZZv1q7Qytc26btmmZaVmN225mbt5K/dyV8IVXFGRHQaYef/+gBkZBpCBmTkw83o+nIfMmbO8z4eBeXHO53yOSkQERERERA6gVroAIiIich8MFkREROQwDBZERETkMAwWRERE5DAMFkREROQwDBZERETkMAwWRERE5DAMFkREROQwDBZERETkMAwWbkKlUuHxxx9XugzyICUlJXjuuecQGxsLtVqNYcOGKV1SpRYvXgyVSoWTJ08qXUq13n77bTRr1gwajQY33HCD0uXUycmTJ6FSqbB48WKlS7mm+Ph43H777UqX4VYYLBSkUqlq9Ni8ebPSpRLZ+Oyzz/D2229j+PDh+Pzzz/HUU08pWs+bb76JVatWKVpDba1duxbPPfccevTogUWLFuHNN99UuiS3cujQIUydOrXeh0t34aV0AZ5syZIlVs+/+OILrFu3zmb69ddf78qyiGpk48aNaNy4MWbPnq10KQBKg8Xw4cNtjpyMHj0a999/P3Q6nTKF1cDGjRuhVqvx6aefQqvVKl2O2zl06BCmTZuGvn37Ij4+Xuly3B6DhYIeeOABq+fbtm3DunXrbKaTc+Xl5cHf31/pMhqc9PR0BAcHK13GNWk0Gmg0GqXLqFZ6ejp8fX0ZKsgt8FRIPZeXl4fJkycjNjYWOp0OLVu2xDvvvIOa3JT2jTfegFqtxpw5cyzTVq9ejV69esHf3x+BgYEYMmQIDh48aLXc2LFjERAQgDNnzmDYsGEICAhAREQEnnnmGRiNxhrVvXr1avTp0weBgYEICgpCly5d8PXXX1vNs2LFCnTq1Am+vr4IDw/HAw88gDNnzlhef+edd6BSqXDq1Cmb9U+ZMgVarRZXrlyxTNu+fTsGDRoEvV4PPz8/9OnTB7///rvVclOnToVKpcKhQ4fwj3/8AyEhIejZsycAYN++fRg7diyaNWsGHx8fREdH46GHHsLly5dttr9582Z07twZPj4+SExMxMKFCy3rrujLL7+07GdoaCjuv/9+pKWlXbMNT506hcceewwtW7aEr68vwsLCMGLECJvDucXFxZg2bRpatGgBHx8fhIWFoWfPnli3bl2168/IyMAzzzyDdu3aISAgAEFBQRg8eDD27t1b7XLm8+ebNm3CwYMHrU7Zbd68udLTd5Wdc7fnfWYymfDBBx+gXbt28PHxQUREBAYNGoRdu3YBKD2tmJeXh88//9xSz9ixYwFU3cdi/vz5aNOmDXQ6HWJiYjBhwgRkZmZazdO3b1+0bdsWhw4dQr9+/eDn54fGjRvjrbfeqraNzEpKSvD6668jMTEROp0O8fHxePHFF2EwGCzzqFQqLFq0CHl5eZbaq+ubYK5p37596NOnD/z8/NC8eXN8++23AIBff/0V3bp1g6+vL1q2bIn169dbLV+T95WIoF+/foiIiEB6erplelFREdq1a4fExETk5eXVqA3KO3LkCIYPH47Q0FD4+Pigc+fO+OGHH6zmMX+/fv/9dzz99NOIiIiAv78/7rrrLly8eNFqXpPJhKlTpyImJgZ+fn7o168fDh06hPj4eKvv/4gRIwAA/fr1q/IU82+//YauXbvCx8cHzZo1wxdffGH3/lEZoXpjwoQJUv5bYjKZpH///qJSqeThhx+WuXPnytChQwWATJo0yWpZADJhwgTL85deeklUKpV89NFHlmlffPGFqFQqGTRokMyZM0dmzZol8fHxEhwcLCdOnLDMN2bMGPHx8ZE2bdrIQw89JAsWLJB77rlHAMj8+fOvuR+LFi0SlUolbdu2lenTp8u8efPk4YcfltGjR1vNA0C6dOkis2fPlhdeeEF8fX0lPj5erly5IiIip06dEpVKJW+99ZbNNpo1ayZDhgyxPN+wYYNotVrp3r27vPvuuzJ79mxp3769aLVa2b59u2W+V199VQBI69at5c4775T58+fLvHnzRETknXfekV69eslrr70mH330kUycOFF8fX2la9euYjKZLOv4888/RafTSXx8vMycOVOmT58uMTEx0qFDB6n4I/XGG2+ISqWS++67T+bPny/Tpk2T8PBwq/2syooVK6RDhw7yyiuvyEcffSQvvviihISESFxcnOTl5Vnme/HFF0WlUsn48ePl448/lnfffVdGjhwpM2fOrHb9O3fulMTERHnhhRdk4cKF8tprr0njxo1Fr9fLmTNnqlwuNzdXlixZIq1atZImTZrIkiVLZMmSJXL+/HnZtGmTAJBNmzZZLXPixAkBIIsWLbJMs+d9NnbsWAEggwcPlvfff1/eeecdufPOO2XOnDkiIrJkyRLR6XTSq1cvSz1//PGHiFx9r5V/j5vfB0lJSTJnzhx5/PHHRaPRSJcuXaSoqMgyX58+fSQmJkZiY2Nl4sSJMn/+fOnfv78AkJ9//rna9jXvIwAZPny4zJs3Tx588EEBIMOGDbPMs2TJEunVq5fodDpL7ceOHatyneVrevbZZ2XOnDnSunVr0Wg0smzZMomOjpapU6fK+++/b/l+ZmdnW5av6fvq+PHjEhAQIHfddZdl2gsvvCAqlUp+/fXXave7su/3gQMHRK/XS+vWrWXWrFkyd+5c6d27t6hUKvnuu+8s85m/Xx07dpT+/fvLnDlzZPLkyaLRaOTee++12s5zzz0nAGTo0KEyd+5cGT9+vDRp0kTCw8NlzJgxIiJy7NgxefLJJwWAvPjii1bvVxGRuLg4admypURFRcmLL74oc+fOlRtvvFFUKpUcOHCg2v2kyjFY1CMVg8WqVasEgLzxxhtW8w0fPlxUKpUcPXrUMq18sJg8ebKo1WpZvHix5fWcnBwJDg6W8ePHW63r/Pnzotfrraabfxm+9tprVvN27NhROnXqVO0+ZGZmSmBgoHTr1k0KCgqsXjN/OBcVFUlkZKS0bdvWap6ffvpJAMgrr7ximda9e3ebbe7YsUMAyBdffGFZb4sWLWTgwIFWASA/P18SEhLklltusUwzf6CMHDnSpvb8/HybaUuXLhUAsmXLFsu0oUOHip+fn9WHb0pKinh5eVl9/06ePCkajUamT59utc79+/eLl5eXzfSa1LN161arfRcR6dChg1XIqqnCwkIxGo1W006cOCE6nc7me1+ZPn36SJs2baym2RssavI+27hxowCQJ5980qaG8t9vf39/y4dJeRWDRXp6umi1Wrn11lut9n/u3LkCQD777DOrfazY3gaDQaKjo+Wee+6xbZRy9uzZIwDk4Ycftpr+zDPPCADZuHGjVVv4+/tXu76KNX399deWaUeOHBEAolarZdu2bZbpa9assWn3mr6vREQWLlwoAOTLL7+Ubdu2iUajsfmjpjKVfb8HDBgg7dq1k8LCQss0k8kkN998s7Ro0cIyzfz9SkpKsvr+PvXUU6LRaCQzM1NESn93eXl5WYU0EZGpU6cKAKv3wooVKyp9X4qUBouKP+Pp6emi0+lk8uTJ19xXssVTIfXYzz//DI1GgyeffNJq+uTJkyEiWL16tdV0EcHjjz+ODz74AF9++SXGjBljeW3dunXIzMzEyJEjcenSJctDo9GgW7du2LRpk832H3nkEavnvXr1wvHjx6uted26dcjJycELL7wAHx8fq9fMpwl27dqF9PR0PPbYY1bzDBkyBK1atcJ///tfy7T77rsPu3fvxrFjxyzTli9fDp1OhzvvvBMAsGfPHqSkpOAf//gHLl++bNm3vLw8DBgwAFu2bIHJZKp23wDA19fX8nVhYSEuXbqEm266CQDw559/AgCMRiPWr1+PYcOGISYmxjJ/8+bNMXjwYKv1fffddzCZTLj33nut2jw6OhotWrSotM2rqqe4uBiXL19G8+bNERwcbKkHAIKDg3Hw4EGkpKRUu76KdDod1Gq1Zb8uX76MgIAAtGzZ0mr9znat99l//vMfqFQqvPrqqzbLVnbq6VrWr1+PoqIiTJo0ybL/ADB+/HgEBQVZvf8AICAgwKrfk1arRdeuXa/5s/Dzzz8DAJ5++mmr6ZMnTwYAm+3YIyAgAPfff7/lecuWLREcHIzrr78e3bp1s0w3f12+1pq+rwDgX//6FwYOHIgnnngCo0ePRmJiYq2uWMnIyMDGjRtx7733Iicnx/KzcPnyZQwcOBApKSlWp0HN2y7//e3VqxeMRqPl1OiGDRtQUlKCxx57zGq5J554wu76WrdujV69elmeR0REoGXLltf8HlPlFAsWW7ZswdChQxETEwOVSlWry8REBO+88w6uu+466HQ6NG7cGNOnT3d8sQo5deoUYmJiEBgYaDXdfJVIxb4HX3zxBebNm4c5c+Zg5MiRVq+ZP3T69++PiIgIq8fatWutzqMCsJzHLi8kJMSqT0NlzAGgbdu21e4XUPrLsKJWrVpZ7deIESOgVquxfPlyAKXf8xUrVmDw4MEICgqy2rcxY8bY7Nsnn3wCg8GArKwsq+0kJCTYbDsjIwMTJ05EVFQUfH19ERERYZnPvHx6ejoKCgrQvHlzm+UrTktJSYGIoEWLFjZ1HT582KbNKyooKMArr7xi6V8THh6OiIgIZGZmWu3Pa6+9hszMTFx33XVo164dnn32Wezbt6/adQOl56dnz56NFi1aWK1/3759Nu3lLDV5nx07dgwxMTEIDQ11yDarev9ptVo0a9bM5ueqSZMmNgGmJj8Lp06dglqttnlfREdHIzg4uNK+QzVVWU16vR6xsbE20wBY1VrT95XZp59+ivz8fKSkpGDx4sVWwaSmjh49ChHByy+/bPOzYA6MFX8emjZtavU8JCTEal/M7VexfUNDQy3z1lTFbZm3d63vMVVOsatC8vLy0KFDBzz00EO4++67a7WOiRMnYu3atXjnnXfQrl07ZGRkICMjw8GVNhw9evTAnj17MHfuXNx7771Wv4jNf7EvWbIE0dHRNst6eVm/FepLL/qYmBj06tUL33zzDV588UVs27YNqampmDVrlmUe8769/fbbVQ4sFBAQYPW8sl+O9957L/744w88++yzuOGGGxAQEACTyYRBgwbZHPGoCZPJBJVKhdWrV1fanhVrquiJJ57AokWLMGnSJHTv3h16vR4qlQr333+/VT29e/fGsWPH8P3332Pt2rX45JNPMHv2bHz44Yd4+OGHq1z/m2++iZdffhkPPfQQXn/9dYSGhkKtVmPSpEm12l+g6iMIVXX6rS/vs+pUVaPUoAM1ULujKtdSVU01qbWm7yuzzZs3Wzqb7t+/H927d7e7XvN6n3nmGQwcOLDSeSoGhLq2uz1cuS1PoFiwGDx4sM2h4/IMBgNeeuklLF26FJmZmWjbti1mzZqFvn37AgAOHz6MBQsW4MCBA5a/PCr7K7Qhi4uLw/r165GTk2N11OLIkSOW18tr3rw53nrrLfTt2xeDBg3Chg0bLMslJiYCACIjI5GUlOS0ms3bOXDgQKV/1ZevOzk5Gf3797d6LTk52Wa/7rvvPjz22GNITk7G8uXL4efnh6FDh9psMygoqNb7duXKFWzYsAHTpk3DK6+8Yple8fRCZGQkfHx8cPToUZt1VJyWmJgIEUFCQgKuu+46u2v69ttvMWbMGLz77ruWaYWFhTZXLgClf6WNGzcO48aNQ25uLnr37o2pU6dWGyy+/fZb9OvXD59++qnV9MzMTISHh9tdL3D1r8qKNdblr/PExESsWbMGGRkZ1R61qOkHePn3X7NmzSzTi4qKcOLECYf9fMTFxcFkMiElJcVqLJoLFy4gMzPT5n3uKva8r86dO4cnnngCt956K7RarSUY2Fu7uZ29vb0d2r5A6c9d+d/9ly9ftjnS4IxwR1Wrt30sHn/8cWzduhXLli3Dvn37MGLECAwaNMjyi/7HH39Es2bN8NNPPyEhIQHx8fF4+OGH3eqIxW233Qaj0Yi5c+daTZ89ezZUKlWlwax9+/b4+eefcfjwYQwdOhQFBQUAgIEDByIoKAhvvvkmiouLbZareBlXbd16660IDAzEjBkzUFhYaPWaOf137twZkZGR+PDDD60uu1u9ejUOHz6MIUOGWC13zz33QKPRYOnSpVixYgVuv/12q3EnOnXqhMTERLzzzjvIzc2t1b6Z/2Kp+BfK+++/bzNfUlISVq1ahbNnz1qmHz161KbPy9133w2NRoNp06bZrFdEKr2MteK2Ki43Z84cm7/+K64nICAAzZs3t2rbmq5/xYoVNue67REXFweNRoMtW7ZYTZ8/f36t13nPPfdARDBt2jSb18rX7+/vX+mHY0VJSUnQarX497//bbX8p59+iqysLJv3X23ddtttAGzfQ++99x4AOGw79qrp+woo7XdiMpnw6aef4qOPPoKXlxf++c9/2v2XfGRkJPr27YuFCxfi3LlzNq/X5vfPgAED4OXlhQULFlhNr/j7EoDl90VN3h9Ud/VygKzU1FQsWrQIqamplg5yzzzzDH755RfLcLfHjx/HqVOnsGLFCnzxxRcwGo146qmnMHz4cGzcuFHhPXCMoUOHol+/fnjppZdw8uRJdOjQAWvXrsX333+PSZMmWf5Sr+imm27C999/j9tuuw3Dhw/HqlWrEBQUhAULFmD06NG48cYbcf/99yMiIgKpqan473//ix49elT6A2mvoKAgzJ49Gw8//DC6dOliGSti7969yM/Px+effw5vb2/MmjUL48aNQ58+fTBy5EhcuHABH3zwAeLj422Gho6MjES/fv3w3nvvIScnB/fdd5/V62q1Gp988gkGDx6MNm3aYNy4cWjcuDHOnDmDTZs2ISgoCD/++OM16+7duzfeeustFBcXo3Hjxli7di1OnDhhM+/UqVOxdu1a9OjRA48++qgl/LVt2xZ79uyxzJeYmIg33ngDU6ZMwcmTJzFs2DAEBgbixIkTWLlyJf71r3/hmWeeqbKm22+/HUuWLIFer0fr1q2xdetWrF+/HmFhYVbztW7dGn379kWnTp0QGhqKXbt24dtvv73mvWNuv/12vPbaaxg3bhxuvvlm7N+/H1999ZXVX/H20uv1GDFiBObMmQOVSoXExET89NNP1+xPUp1+/fph9OjR+Pe//42UlBTLqan//e9/6Nevn2U/O3XqhPXr1+O9995DTEwMEhISrDoymkVERGDKlCmYNm0aBg0ahDvuuAPJycmYP38+unTp4rAB6jp06IAxY8bgo48+QmZmJvr06YMdO3bg888/x7Bhw9CvXz+HbMdeNX1fLVq0CP/973+xePFiNGnSBEBpAHnggQewYMECm06T1zJv3jz07NkT7dq1w/jx49GsWTNcuHABW7duxenTp685fkpFUVFRmDhxIt59913ccccdGDRoEPbu3YvVq1cjPDzc6ijFDTfcAI1Gg1mzZiErKws6nQ79+/dHZGSkXdukGnLlJShVASArV660PDdfdujv72/18PLyslzHPH78eAEgycnJluV2794tAOTIkSOu3gWHqHi5qUjpZaJPPfWUxMTEiLe3t7Ro0ULefvttq8uwRGzHsRAR+f7778XLy0vuu+8+y2V1mzZtkoEDB4perxcfHx9JTEyUsWPHyq5duyzLVXXpm/lSzZr44Ycf5OabbxZfX18JCgqSrl27ytKlS63mWb58uXTs2FF0Op2EhobKqFGj5PTp05Wu7+OPPxYAEhgYaHMZq9lff/0ld999t4SFhYlOp5O4uDi59957ZcOGDTb7cPHiRZvlT58+LXfddZcEBweLXq+XESNGyNmzZwWAvPrqq1bzbtiwQTp27CharVYSExPlk08+kcmTJ4uPj4/Nev/zn/9Iz549Le/jVq1ayYQJE6zeu5W5cuWKjBs3TsLDwyUgIEAGDhwoR44ckbi4OKtL6d544w3p2rWrBAcHi6+vr7Rq1UqmT59uNR5DZQoLC2Xy5MnSqFEj8fX1lR49esjWrVulT58+0qdPn2qXFan8clMRkYsXL8o999wjfn5+EhISIv/3f/8nBw4cqPRy05q+z0pKSuTtt9+WVq1aiVarlYiICBk8eLDs3r3bMs+RI0ekd+/e4uvra3W5YWXjWIiUXl7aqlUr8fb2lqioKHn00Udtxhapah/HjBkjcXFx1TeQiBQXF8u0adMkISFBvL29JTY2VqZMmWJ1yWV1bVGZqmqKi4ur9LLjir8bavK+SktLE71eL0OHDrVZ31133SX+/v5y/PjxKmus7HJTkdIxJR588EGJjo4Wb29vady4sdx+++3y7bffWuYxf7927txptWxllzKXlJTIyy+/LNHR0eLr6yv9+/eXw4cPS1hYmDzyyCNWy3/88cfSrFkz0Wg0Vuupqt1q+nNAtlQiyvdOUalUWLlypWWM/+XLl2PUqFE4ePCgTaeagIAAREdH49VXX7U5rF9QUAA/Pz+sXbsWt9xyiyt3gQjDhg2r1WWfROQ4mZmZCAkJwRtvvIGXXnpJ6XI8Ur08FdKxY0cYjUakp6dbXVtcXo8ePVBSUoJjx45ZTgn8/fffAGw7NRI5WkFBgdWVJSkpKfj555+txg4hIueq+HMIXO3TYu7oT66n2BGL3NxcSy/6jh074r333kO/fv0QGhqKpk2b4oEHHsDvv/+Od999Fx07dsTFixexYcMGtG/fHkOGDIHJZEKXLl0QEBCA999/HyaTCRMmTEBQUBDWrl2rxC6RB2nUqJHlviKnTp3CggULYDAY8Ndff6FFixZKl0fkERYvXozFixfjtttuQ0BAAH777TcsXboUt956K9asWaN0eZ5LqXMw5vNlFR/mc3xFRUXyyiuvSHx8vHh7e0ujRo3krrvukn379lnWcebMGbn77rslICBAoqKiZOzYsXL58mWF9og8ydixYyUuLk50Op0EBQXJwIEDrc73E5Hz7d69WwYMGCBhYWHi7e0tTZo0kYkTJ0pOTo7SpXm0etHHgoiIiNxDvR3HgoiIiBoeBgsiIiJyGJdfFWIymXD27FkEBgZymFUiIqIGQkSQk5ODmJgYqzsDV+TyYHH27FmbO/ARERFRw5CWlmYZjbUyLg8W5ptipaWlWW57TQ1AXh5QNrw6zp4Fyt2rg4iI3F92djZiY2OtbopZGZcHC/Ppj6CgIAaLhqT8CKhBQQwWREQe6lrdGNh5k4iIiByGwYKIiIgchsGCiIiIHIbBgoiIiByGwYKIiIgchsGCiIiIHIbBgoiIiByGwYKIiIgchsGCiIiIHIbBgoiIiByGwYKIiIgchsGCiIiIHMblNyEjouqJCEpMAqNJYBKBSQCTCMQEGMU8TSACyzzlvzZJ6TqMIjCZcPV1y3Kl85jnR+k/iHk7ZTUIUPZa6fJls1rWB/N0mF8r/1ysp5WfbvW67bIom8dkknKvly6DCvNb2symDcu/JlW+VlnbV72eqrdRcTu2r1W9YF1ql2pfq7r4apqgViq2W93X59DVOWF/Hbw+h1cIPH3LdQj08Xb4emuCwYI8lskkKCwxIs9gRH5RieV/Q4kJRSUmGEqMMJSYrj6KjSgymmAoNtnMY/66xFgaCkpMpnJfC0qM5uemsuelXxtNgmJj2eumq4GCiKguHu2byGBBZA8RQY6hBFn5xcgqsH5klpuWXVCM7MJi5BcZyx5XA0R+kVHp3ag1lQrQqFRQq1RQqQC1SgWN2vprtar09sal85V+rVbbLqdSASqU/a9SQVW2fqvpZRtVl31tNR9KJ5qfV1wnKsyvLvc1yq2//DJqVfl1lt/W1ecwb7tcm1i1UYX2sn7NekL5123vCF2zbdisp7pt2CxX9W2o7aq9unoqrKj6G1/XgoNXWHE/67w+h9fn4PU5eIV+WuU+3hksqF4REWTkFeFMZgHOXCnAxVwDLuYYcKns/4u5RbiUY8DFXAOKSkwO266/VgM/nRf8tBr4eGmg81ZDq1FD562GzktT7ms1tF6l0yr72lujgkZt/l8FL7UaXmoVvDRlX2tUZc8rTK9kHrW6NBRYBQbV1QBARFQfMViQyxUbTUjLyMexi3k4fjEXaVfycfpKaZA4faUABcU1P5Kg81JD7+sNva83gv1K/w8qe25+BPp4I0CngZ/WC/7m/7Ve8NVq4K8rDRJqNT+oiYgcgcGCnEZEkJZRgANns3DwbBZSLuTi2MVcnLqcj5Jr9COICtIhJtgXUYE+iAjUITxAV/a/1vI8PEAHX63GRXtDREQ1wWBBDnM2swA7T2Zg/+mssjCRjZzCkkrn9fXWIDHSH83CAxAX5ofGwb5oEuKHJiG+aBTsA50XAwMRUUPEYEG1IiI4cSkPW49fxq6TV7DjRAbOZBbYzKfVqNGqUSDaxAShZVQgEiMDkBgRgOggH55+ICJyQwwWZLfXfjyI9al5SM3It5quVgFtYvTo2DQYbRvr0TZGjxZRAfDWcBw2IiJPwWBB1co1lGDD4QtYu/M45pVNW7ojDQVaH3hrVLixaQi6JYSiS0IoOjYNQYCObykiIk/GTwGyUWI0YXPyRXy7+zQ2JafDUGKCb1Gh5fX7uzbBze3jcXNiGPwZJIiIqBx+KpDF2cwCLNuZhm92puF89tUgkRDuj2HXxQCzS5+/OrQt4O+vUJVERFSfMVgQ/r6Qgw83H8P3e89ahpMO9dfinhsbY1jHxmjdKAiq/PxrrIWIiIjBwqP9fSEH76xJxtpDFyzTuiWE4oGb4nBrmyhe8klERHZjsPBA6dmFeG/d3/hmVxpMUjpG/aA20XikTyI6xAYrXR4RETVgDBYexGQSfLX9FGb9koxcQ+nAVYPaROOZgS3RPDJA4eqIiMgdMFh4iKPpOXju2334MzUTANAhNhgvD7keneNDlS2MiIjcCoOFmxMRfLMrDa/+cBCFxSYE6Lzw3KCWGNUtDhqOfElERA7GYOHGcg0lmPLdfvy49ywAoFeLcLw1vD0a6X0VroyIiNwVg4WbOpNZgH8u3okj53OgUavwzK0t8X+9m/H+HERE5FQMFm5ob1om/vn5LlzKNSA8QIeFo29Epzj2pSAiIudjsHAz245fxkOLdyK/yIhW0YH4dGwXNA7mqQ8iInINBgs38r+Uixj/xS4UFpvQs3k4PhzdiTcFIyIil+Knjpv449gl/PPzXSgqMaFfywgseKATfLw5ciYREbkWg4UbOHg2C//6YjeKSky4pXUU5v3jRmi91EqXRUREHoifPg1cWkY+xi7aiVxDCbomhGLOyI4MFUREpBh+AjVg+UUlePjzXbiYY0DLqEB8/GBnnv4gIiJFMVg0UCKC5/+zH8kXchARqMPih7pA7+utdFlEROThGCwaqM9+P4kf956Fl1qF+aNu5GiaRERULzBYNECHz2Vj1uojAID/N+R6dOGNxIiIqJ5gsGhgDCVGPLV8D4qMJiRdH4kxN8crXRIREZGF3cHizJkzeOCBBxAWFgZfX1+0a9cOu3btckZtVInZ61Jw5HwOwvy1mHF3e6hUvPcHERHVH3aNY3HlyhX06NED/fr1w+rVqxEREYGUlBSEhIQ4qz4qZ09aJhZuOQYAePPudogI1ClcERERkTW7gsWsWbMQGxuLRYsWWaYlJCQ4vCiyZTQJ/t+q/RABht0Qg4FtopUuiYiIyIZdp0J++OEHdO7cGSNGjEBkZCQ6duyIjz/+uNplDAYDsrOzrR5kv6+2n8KBM9kI8vHCS0NaK10OERFRpewKFsePH8eCBQvQokULrFmzBo8++iiefPJJfP7551UuM2PGDOj1essjNja2zkV7mos5Bry9JhkA8OygVjwFQkRE9ZZKRKSmM2u1WnTu3Bl//PGHZdqTTz6JnTt3YuvWrZUuYzAYYDAYLM+zs7MRGxuLrKwsBAUF1aF0z/H/Vu3Hl9tS0a6xHqsm9IBGrUCHzbw8ICCg9OvcXMDf3/U1EBGRYrKzs6HX66/5+W3XEYtGjRqhdWvrw/DXX389UlNTq1xGp9MhKCjI6kE1d+xiLpbuSAMAvDTkemVCBRERUQ3ZFSx69OiB5ORkq2l///034uLiHFoUXfX2L8kwmgQDWkXipmZhSpdDRERULbuCxVNPPYVt27bhzTffxNGjR/H111/jo48+woQJE5xVn0f7M/UKfjl4HmoV8PzgVkqXQ0REdE12BYsuXbpg5cqVWLp0Kdq2bYvXX38d77//PkaNGuWs+jzavzekAADuubEJrosKVLgaIiKia7NrHAsAuP3223H77bc7oxYqZ9/pTGxOvgi1CpjQr7nS5RAREdUI7xVST83ZeBQAcOcNjREfziswiIioYWCwqIcOn8vGukMXoOLRCiIiamAYLOqhT/53AgBwW7tGaB4ZoHA1RERENcdgUc9czDHgx71nAQAP9+R9WIiIqGFhsKhnvt6eiiKjCTfEBqNjU941loiIGhYGi3rEUGLEkm2nAADjesQrWwwREVEtMFjUI//ddw6Xcg2ICtLhtnaNlC6HiIjIbgwW9cjnf5wEAIy+KQ7eGn5riIio4eGnVz1x+Fw29p7OgrdGhfu7NlW6HCIiolphsKgnlu8svYNp0vVRCA/QKVwNERFR7TBY1AOGEiNW7TkDALi3S6zC1RAREdUeg0U9sPbgBWTmF6OR3ge9W0QoXQ4REVGtMVjUA9/sKj0NMrxTE2jUKoWrISIiqj0GC4WdvpKP345eAgCM6MTTIERE1LAxWCjsx73nIALc1CwUTcP8lC6HiIioThgsFPZD2X1B7ryhscKVEBER1R2DhYKOpufg8LlseKlVGNw2WulyiIiI6ozBQkE/7D0HAOh9XQSC/bQKV0NERFR3DBYKERHL7dHv6BCjcDVERESOwWChkINns3HiUh50XmoktY5SuhwiIiKHYLBQiPloRdL1UQjQeSlcDRERkWMwWChARPDLwfMAgCHteXt0IiJyHwwWCki+kINTl/Oh81Kjb0sO4U1ERO6DwUIBaw9eAAD0ahEOPy1PgxARkftgsFDA2kOlp0Fubc2xK4iIyL0wWLjYmcwCHDiTDbUKGHB9pNLlEBERORSDhYutK+u02TkuFGEBOoWrISIiciwGCxdbe6i0f8WtbTh2BRERuR8GCxfKyi/G9hMZAIBbOCgWERG5IQYLF/rf0YswmgQtIgMQF+avdDlEREQOx2DhQpuTLwIAx64gIiK3xWDhIiaT4Ne/S4NFn+t4NQgREbknBgsXOXQuGxdzDPDTatAlIUTpcoiIiJyCwcJFzEcrbk4Mg85Lo3A1REREzsFg4SKbk9MBAH1a8jQIERG5LwYLF8gqKMafqZkAgL7XseMmERG5LwYLF/gt5RKMJkFihD9iQ/2ULoeIiMhpGCxc4LejvBqEiIg8A4OFC/x+9DIAoGeLMIUrISIici4GCydLy8hHakY+NGoVusSHKl0OERGRU9kVLKZOnQqVSmX1aNWqlbNqcwtbj5UerejQRI9AH2+FqyEiInIuL3sXaNOmDdavX391BV52r8Kj/H7sEgCgR/NwhSshIiJyPrtTgZeXF6Kjo2s8v8FggMFgsDzPzs62d5MNlojgj7IjFt0T2b+CiIjcn919LFJSUhATE4NmzZph1KhRSE1NrXb+GTNmQK/XWx6xsbG1LrahOZqei4s5Bui81LixKYfxJiIi92dXsOjWrRsWL16MX375BQsWLMCJEyfQq1cv5OTkVLnMlClTkJWVZXmkpaXVueiG4vejpadBusSHwsebw3gTEZH7s+tUyODBgy1ft2/fHt26dUNcXBy++eYb/POf/6x0GZ1OB51OV7cqGyieBiEiIk9Tp8tNg4ODcd111+Ho0aOOqsdtmEyCbcdLg8XNDBZEROQh6hQscnNzcezYMTRq1MhR9biN5As5yC4sgZ9Wg3aN9UqXQ0RE5BJ2BYtnnnkGv/76K06ePIk//vgDd911FzQaDUaOHOms+hqsnSczAAA3Ng2Bl4bjkBERkWewq4/F6dOnMXLkSFy+fBkRERHo2bMntm3bhogI3rGzoh0nSoMFR9skIiJPYlewWLZsmbPqcCsiYjli0SWBl5kSEZHn4DF6Jzh9pQAXsg3w1qjQMZbBgoiIPAeDhROYT4O0bayHr5bjVxARkedgsHAC82mQruxfQUREHobBwgl2nGTHTSIi8kwMFg52KdeA4xfzAACd49m/goiIPAuDhYPtOnkFAHBdVACC/bQKV0NERORaDBYO9ldaabDoFMejFURE5HkYLBxsT2omAPAyUyIi8kgMFg5kNAn2n8kCANzQNFjZYoiIiBTAYOFAf1/IQX6REQE6LyRGBChdDhERkcsxWDjQX2WnQdo30UOjVilbDBERkQIYLBxoT1nHzRtig5UthIiISCEMFg60Jy0TAIMFERF5LgYLB8kpLEZKei4AdtwkIiLPxWDhIPtPZ0EEaBzsi8hAH6XLISIiUgSDhYP8ZT4NwqMVRETkwRgsHOQvy8BYwYrWQUREpCQGCwc5UDYwVvsmwcoWQkREpCAGCwe4lGvA+exCAEDrmCCFqyEiIlIOg4UDHDybDQBoFu6PAJ2XwtUQEREph8HCAQ6eLT0NwqMVRETk6RgsHODgmdIjFm0b6xWuhIiISFkMFg5gPmLRhkcsiIjIwzFY1FF2YTFOXs4HALSJ4RELIiLybAwWdXSorONmjN4Hof5ahashIiJSFoNFHZmvCGnD/hVEREQMFnV1sGxgrLY8DUJERMRgUVeWIxbsuElERMRgUReFxUYcvVh6q3ReakpERMRgUSfJ53NgNAnC/LWICtIpXQ4REZHiGCzqIPl8DgCgVaNAqFQqhashIiJSHoNFHSRfKA0W10UFKlwJERFR/cBgUQd/lwWLlgwWREREABgs6uRI2amQltEMFkRERACDRa1l5BXhYo4BANCCRyyIiIgAMFjUmrnjZpMQXwTovBSuhoiIqH5gsKglc/+KVjwNQkREZMFgUUu8IoSIiMgWg0UtJbPjJhERkY06BYuZM2dCpVJh0qRJDiqnYRAR/M1gQUREZKPWwWLnzp1YuHAh2rdv78h6GoRzWYXIMZTAS61Cs/AApcshIiKqN2oVLHJzczFq1Ch8/PHHCAkJcXRN9Z75NEhCuD+0XjybREREZFarT8UJEyZgyJAhSEpKuua8BoMB2dnZVo+Gztxxk6dBiIiIrNk9AMOyZcvw559/YufOnTWaf8aMGZg2bZrdhdVnf/OKECIiokrZdcQiLS0NEydOxFdffQUfH58aLTNlyhRkZWVZHmlpabUqtD45djEPANAikv0riIiIyrPriMXu3buRnp6OG2+80TLNaDRiy5YtmDt3LgwGAzQajdUyOp0OOp3OMdXWAyKCY+m5AIBEBgsiIiIrdgWLAQMGYP/+/VbTxo0bh1atWuH555+3CRXuKD3HgFxDCdQqIC7MT+lyiIiI6hW7gkVgYCDatm1rNc3f3x9hYWE2092V+WhF01A/6LzcP0gRERHZg9dK2unYxbLTIBE8DUJERFRRnW/LuXnzZgeU0XCYO242Z/8KIiIiGzxiYScesSAiIqoag4Wdrl4R4q9wJURERPUPg4Ud8gwlOJtVCAC8RwgREVElGCzscOJSaf+KMH8tQvy1CldDRERU/zBY2IH9K4iIiKrHYGEH9q8gIiKqHoOFHcyXmvKIBRERUeUYLOxgORXCMSyIiIgqxWBRQ0aT4HhZ583mPGJBRERUKQaLGjp9JR9FJSbovNSICfZVuhwiIqJ6icGihsynQRLC/aFRqxSuhoiIqH5isKih4+y4SUREdE0MFjV08nJpsIgP91O4EiIiovqLwaKGTl7KBwDEh3EMCyIioqowWNSQeTjv+HAGCyIioqowWNRAYbERZ7MKAPCIBRERUXUYLGrg9JV8iAABOi+EB/DmY0RERFVhsKiBE+b+FeF+UKl4qSkREVFVGCxq4KS5fwVPgxAREVWLwaIGTpRdaprAjptERETVYrCoAfMRizgesSAiIqoWg0UNnLpc2scigYNjERERVYvB4hp4qSkREVHNMVhcQ2pG6aWmgT5eCPXnpaZERETVYbC4BvOImwnh/rzUlIiI6BoYLK6Bl5oSERHVHIPFNVjuahrGjptERETXwmBxDZa7mnIMCyIiomtisLgGyxELBgsiIqJrYrCoRkGREeeyCgEACexjQUREdE0MFtU4lVF6tELv640QXmpKRER0TQwW1bh6RQg7bhIREdUEg0U1Tl5mx00iIiJ7MFhUIzWjNFjEhfKIBRERUU0wWFQjrSxYxDJYEBER1QiDRTXMRyyaMlgQERHVCINFFUqMJpy5UnpX06bsvElERFQjDBZVOJdViBKTQKtRIyrQR+lyiIiIGgQGiyqY+1c0CfWFWs27mhIREdWEXcFiwYIFaN++PYKCghAUFITu3btj9erVzqpNUexfQUREZD+7gkWTJk0wc+ZM7N69G7t27UL//v1x55134uDBg86qTzEMFkRERPbzsmfmoUOHWj2fPn06FixYgG3btqFNmzYOLUxpaeaOmwwWRERENWZXsCjPaDRixYoVyMvLQ/fu3aucz2AwwGAwWJ5nZ2fXdpMulcoxLIiIiOxmd+fN/fv3IyAgADqdDo888ghWrlyJ1q1bVzn/jBkzoNfrLY/Y2Ng6FewqlsGxQhgsiIiIasruYNGyZUvs2bMH27dvx6OPPooxY8bg0KFDVc4/ZcoUZGVlWR5paWl1KtgVcgqLkZFXBACIDfVVuBoiIqKGw+5TIVqtFs2bNwcAdOrUCTt37sQHH3yAhQsXVjq/TqeDTqerW5UulpZR2r8i1F+LQB9vhashIiJqOOo8joXJZLLqQ+EO2L+CiIioduw6YjFlyhQMHjwYTZs2RU5ODr7++mts3rwZa9ascVZ9ikjjpaZERES1YlewSE9Px4MPPohz585Br9ejffv2WLNmDW655RZn1aeIq2NYsH8FERGRPewKFp9++qmz6qhXODgWERFR7fBeIZVIYx8LIiKiWmGwqMBkEpzmqJtERES1wmBRwYWcQhQZTfBSq9BIzz4WRERE9mCwqCD1culpkMYhvtDwdulERER2YbCogB03iYiIao/BogJ23CQiIqo9BosKeMSCiIio9hgsKmCwICIiqj0GiwpSM3ipKRERUW0xWJSTX1SCS7mlN1RjHwsiIiL7MViUY75dut7XG3pf3i6diIjIXgwW5fCupkRERHXDYFFOquVSU464SUREVBsMFuWkcgwLIiKiOmGwKIenQoiIiOqGwaIcjmFBRERUNwwWZUSEwYKIiKiOGCzKXMwxwFBigloFxASz8yYREVFtMFiUMR+tiAn2hbeGzUJERFQb/AQtw9MgREREdcdgUSaN9wghIiKqMwaLMhzDgoiIqO4YLMqkMVgQERHVGYNFGfaxICIiqjsGCwCFxUaczy4EwGBBRERUFwwWAE5fKe24GaDzQogfb5dORERUWwwWsO5foVKpFK6GiIio4WKwQPn+FRxxk4iIqC4YLMCOm0RERI7CYAEGCyIiIkdhsADHsCAiInIUjw8WIsJgQURE5CAeHywy8oqQV2SESgU05u3SiYiI6sTjg4W5f0V0kA98vDUKV0NERNSwMVjwNAgREZHDeHywSOMVIURERA7j8cGCl5oSERE5DoMFgwUREZHDeHywSMsovQEZ+1gQERHVnUcHi6ISE85mlQYLHrEgIiKqO7uCxYwZM9ClSxcEBgYiMjISw4YNQ3JysrNqc7qzmQUQAXy9NQgP0CpdDhERUYNnV7D49ddfMWHCBGzbtg3r1q1DcXExbr31VuTl5TmrPqe6eqmpL2+XTkRE5ABe9sz8yy+/WD1fvHgxIiMjsXv3bvTu3duhhbkCO24SERE5ll3BoqKsrCwAQGhoaJXzGAwGGAwGy/Ps7Oy6bNKheI8QIiIix6p1502TyYRJkyahR48eaNu2bZXzzZgxA3q93vKIjY2t7SYdjkcsiIiIHKvWwWLChAk4cOAAli1bVu18U6ZMQVZWluWRlpZW2006HIMFERGRY9XqVMjjjz+On376CVu2bEGTJk2qnVen00Gn09WqOGcSEaReZrAgIiJyJLuChYjgiSeewMqVK7F582YkJCQ4qy6nyyooRo6hBADQJITBgoiIyBHsChYTJkzA119/je+//x6BgYE4f/48AECv18PX19cpBTqL+TRIZKAOvlreLp2IiMgR7OpjsWDBAmRlZaFv375o1KiR5bF8+XJn1ec07F9BRETkeHafCnEXvEcIERGR43nsvUJSOYYFERGRw3lssEjjqRAiIiKH89hgwT4WREREjueRwaLEaMKZTN4unYiIyNE8MlicyyqE0STQeqkRGVj/Bu8iIiJqqDwyWFg6bob4Qq3m7dKJiIgcxaODBU+DEBERORaDBRERETmMRwcLjmFBRETkWB4ZLE4zWBARETmFRwYLngohIiJyDo8LFtmFxbiSXwyARyyIiIgczeOChXko7zB/LQJ0dt2DjYiIiK7BY4MFj1YQERE5nscFC/avICIich4GCyIiInIYDwwWvPkYERGRs3hcsGAfCyIiIufxqGBhNAlOXzEHC1+FqyEiInI/HhUsLmQXotgo8FKr0EjPYEFERORoHhUszB03m4T4QsPbpRMRETmcRwYL9q8gIiJyDo8KFmm81JSIiMipPCpYcAwLIiIi52KwICIiIofxqGDBMSyIiIicy2OCRZ6hBJdyiwAATcMYLIiIiJzBY4JFWtnAWHpfbwT5eCtcDRERkXvynGDBe4QQERE5nccEC3bcJCIicj6PCRbsuElEROR8HhMseMSCiIjI+RgsiIiIyGE8IliYTMLhvImIiFzAI4LFxVwDDCUmaNQqNAr2UbocIiIit+URwcJ8GiQm2AfeGo/YZSIiIkV4xKds6mWeBiEiInIFzwgW5ktNQxgsiIiInMkjgoV5OG+OYUFERORcdgeLLVu2YOjQoYiJiYFKpcKqVaucUJZj8YoQIiIi17A7WOTl5aFDhw6YN2+eM+pxCo5hQURE5Bpe9i4wePBgDB482Bm1OEVhsREXsg0AGCyIiIicze5gYS+DwQCDwWB5np2d7exNWjld1r8iUOeFYD/eLp2IiMiZnN55c8aMGdDr9ZZHbGysszdpJbXczcdUKpVLt01ERORpnB4spkyZgqysLMsjLS3N2Zu0wjEsiIiIXMfpp0J0Oh10Op2zN1Ol1IwCAEDTMAYLIiIiZ3P7cSyuDo7lq3AlRERE7s/uIxa5ubk4evSo5fmJEyewZ88ehIaGomnTpg4tzhHSMjg4FhERkavYHSx27dqFfv36WZ4//fTTAIAxY8Zg8eLFDivMEUTEMuom+1gQERE5n93Bom/fvhARZ9TicJfzipBfZIRKBTTmqRAiIiKnc+s+Fub+FY2CfKDz0ihcDRERkftz62DB/hVERESu5dbBgmNYEBERuZZ7BwvefIyIiMilPCNYcHAsIiIil3DrYME+FkRERK7ltsHCUGLEuexCAEBsCIMFERGRK7htsDhzpQAigK+3BuEBWqXLISIi8ghuGyzSrpTdfIy3SyciInIZtw0WqexfQURE5HJuGyzSeKkpERGRy7ltsLg6OBbvEUJEROQq7hssOIYFERGRy7llsBARngohIiJSgFsGi8z8YuQYSgAATTiGBRERkcu4ZbAwnwaJDNTBx5u3SyciInIVtw4WPA1CRETkWgwWRERE5DBuGSxOX+HgWEREREpwy2DBIxZERETKcO9gwTEsiIiIXMrtgkWx0YSzmaW3S+cRCyIiItdyu2BxLrMQRpNA56VGRIBO6XKIiIg8itsFi/J3NVWrebt0IiIiV3LbYMHTIERERK7ntsEiNoR3NSUiInI1twsWaRkcw4KIiEgpbhcseCqEiIhIOW4XLNKucAwLIiIipbhVsMgqKEZmfjEAIJa3SyciInI5twoW5v4V4QFa+Ou8FK6GiIjI87hlsGDHTSIiImW4VbBgx00iIiJlMVgQERGRw7hlsGDHTSIiImW4VbBgHwsiIiJluU2wMJoEZzILAHAMCyIiIqW4TbA4n12IYqPAW6NCdJCP0uUQERF5JLcJFqmXS0+DNAnxg4a3SyciIlKE2wQL9q8gIiJSXq2Cxbx58xAfHw8fHx9069YNO3bscHRddrt6qSlvl05ERKQUu4PF8uXL8fTTT+PVV1/Fn3/+iQ4dOmDgwIFIT093Rn01xjEsiIiIlGd3sHjvvfcwfvx4jBs3Dq1bt8aHH34IPz8/fPbZZ5XObzAYkJ2dbfVwBgYLIiIi5dkVLIqKirB7924kJSVdXYFajaSkJGzdurXSZWbMmAG9Xm95xMbG1q3iKrCPBRERkfLsChaXLl2C0WhEVFSU1fSoqCicP3++0mWmTJmCrKwsyyMtLa321VZBRPBo30SM6R7HIxZEREQKcvq9xXU6HXQ6nVO3oVKp8HCvZk7dBhEREV2bXUcswsPDodFocOHCBavpFy5cQHR0tEMLIyIioobHrmCh1WrRqVMnbNiwwTLNZDJhw4YN6N69u8OLIyIioobF7lMhTz/9NMaMGYPOnTuja9eueP/995GXl4dx48Y5oz4iIiJqQOwOFvfddx8uXryIV155BefPn8cNN9yAX375xaZDJxEREXkelYiIKzeYnZ0NvV6PrKwsBAUFuXLTVBd5eUBAQOnXubmAv7+y9RARkUvV9PPbbe4VQkRERMpjsCAiIiKHYbAgIiIih2GwICIiIodhsCAiIiKHYbAgIiIih2GwICIiIodhsCAiIiKHcfrdTSsyj8eVnZ3t6k1TXeTlXf06OxswGpWrhYiIXM78uX2tcTVdHixycnIAALGxsa7eNDlKTIzSFRARkUJycnKg1+urfN3lQ3qbTCacPXsWgYGBUKlUDltvdnY2YmNjkZaWxqHCy7BNbLFNbLFNKsd2scU2seVJbSIiyMnJQUxMDNTqqntSuPyIhVqtRpMmTZy2/qCgILf/5tqLbWKLbWKLbVI5tosttoktT2mT6o5UmLHzJhERETkMgwURERE5jNsEC51Oh1dffRU6nU7pUuoNtokttokttknl2C622Ca22Ca2XN55k4iIiNyX2xyxICIiIuUxWBAREZHDMFgQERGRwzBYEBERkcMwWBAREZHDuE2wmDdvHuLj4+Hj44Nu3bphx44dSpfkFDNmzECXLl0QGBiIyMhIDBs2DMnJyVbzFBYWYsKECQgLC0NAQADuueceXLhwwWqe1NRUDBkyBH5+foiMjMSzzz6LkpISV+6K08ycORMqlQqTJk2yTPPENjlz5gweeOABhIWFwdfXF+3atcOuXbssr4sIXnnlFTRq1Ai+vr5ISkpCSkqK1ToyMjIwatQoBAUFITg4GP/85z+Rm5vr6l1xCKPRiJdffhkJCQnw9fVFYmIiXn/9dasbKnlCm2zZsgVDhw5FTEwMVCoVVq1aZfW6o9pg37596NWrF3x8fBAbG4u33nrL2btWa9W1SXFxMZ5//nm0a9cO/v7+iImJwYMPPoizZ89arcPd2qROxA0sW7ZMtFqtfPbZZ3Lw4EEZP368BAcHy4ULF5QuzeEGDhwoixYtkgMHDsiePXvktttuk6ZNm0pubq5lnkceeURiY2Nlw4YNsmvXLrnpppvk5ptvtrxeUlIibdu2laSkJPnrr7/k559/lvDwcJkyZYoSu+RQO3bskPj4eGnfvr1MnDjRMt3T2iQjI0Pi4uJk7Nixsn37djl+/LisWbNGjh49apln5syZotfrZdWqVbJ371654447JCEhQQoKCizzDBo0SDp06CDbtm2T//3vf9K8eXMZOXKkErtUZ9OnT5ewsDD56aef5MSJE7JixQoJCAiQDz74wDKPJ7TJzz//LC+99JJ89913AkBWrlxp9boj2iArK0uioqJk1KhRcuDAAVm6dKn4+vrKwoULXbWbdqmuTTIzMyUpKUmWL18uR44cka1bt0rXrl2lU6dOVutwtzapC7cIFl27dpUJEyZYnhuNRomJiZEZM2YoWJVrpKenCwD59ddfRaT0h8Db21tWrFhhmefw4cMCQLZu3SoipT9EarVazp8/b5lnwYIFEhQUJAaDwbU74EA5OTnSokULWbdunfTp08cSLDyxTZ5//nnp2bNnla+bTCaJjo6Wt99+2zItMzNTdDqdLF26VEREDh06JABk586dlnlWr14tKpVKzpw547zinWTIkCHy0EMPWU27++67ZdSoUSLimW1S8UPUUW0wf/58CQkJsfrZef7556Vly5ZO3qO6qyxsVbRjxw4BIKdOnRIR928TezX4UyFFRUXYvXs3kpKSLNPUajWSkpKwdetWBStzjaysLABAaGgoAGD37t0oLi62ao9WrVqhadOmlvbYunUr2rVrh6ioKMs8AwcORHZ2Ng4ePOjC6h1rwoQJGDJkiNW+A57ZJj/88AM6d+6MESNGIDIyEh07dsTHH39sef3EiRM4f/68VZvo9Xp069bNqk2Cg4PRuXNnyzxJSUlQq9XYvn2763bGQW6++WZs2LABf//9NwBg7969+O233zB48GAAntkmFTmqDbZu3YrevXtDq9Va5hk4cCCSk5Nx5coVF+2N82RlZUGlUiE4OBgA26Qil9/d1NEuXboEo9Fo9YEAAFFRUThy5IhCVbmGyWTCpEmT0KNHD7Rt2xYAcP78eWi1Wssb3iwqKgrnz5+3zFNZe5lfa4iWLVuGP//8Ezt37rR5zRPb5Pjx41iwYAGefvppvPjii9i5cyeefPJJaLVajBkzxrJPle1z+TaJjIy0et3LywuhoaENsk1eeOEFZGdno1WrVtBoNDAajZg+fTpGjRoFAB7ZJhU5qg3Onz+PhIQEm3WYXwsJCXFK/a5QWFiI559/HiNHjrTczdTT26SiBh8sPNmECRNw4MAB/Pbbb0qXoqi0tDRMnDgR69atg4+Pj9Ll1AsmkwmdO3fGm2++CQDo2LEjDhw4gA8//BBjxoxRuDplfPPNN/jqq6/w9ddfo02bNtizZw8mTZqEmJgYj20Tsk9xcTHuvfdeiAgWLFigdDn1VoM/FRIeHg6NRmPTw//ChQuIjo5WqCrne/zxx/HTTz9h06ZNaNKkiWV6dHQ0ioqKkJmZaTV/+faIjo6utL3MrzU0u3fvRnp6Om688UZ4eXnBy8sLv/76K/7973/Dy8sLUVFRHtcmjRo1QuvWra2mXX/99UhNTQVwdZ+q+7mJjo5Genq61eslJSXIyMhokG3y7LPP4oUXXsD999+Pdu3aYfTo0XjqqacwY8YMAJ7ZJhU5qg3c7ecJuBoqTp06hXXr1lmOVgCe2yZVafDBQqvVolOnTtiwYYNlmslkwoYNG9C9e3cFK3MOEcHjjz+OlStXYuPGjTaH1jp16gRvb2+r9khOTkZqaqqlPbp37479+/db/SCYf1Aqfhg1BAMGDMD+/fuxZ88ey6Nz584YNWqU5WtPa5MePXrYXIb8999/Iy4uDgCQkJCA6OhoqzbJzs7G9u3brdokMzMTu3fvtsyzceNGmEwmdOvWzQV74Vj5+flQq61/5Wk0GphMJgCe2SYVOaoNunfvji1btqC4uNgyz7p169CyZcsGecjfHCpSUlKwfv16hIWFWb3uiW1SLaV7jzrCsmXLRKfTyeLFi+XQoUPyr3/9S4KDg616+LuLRx99VPR6vWzevFnOnTtneeTn51vmeeSRR6Rp06ayceNG2bVrl3Tv3l26d+9ued18aeWtt94qe/bskV9++UUiIiIa7KWVlSl/VYiI57XJjh07xMvLS6ZPny4pKSny1VdfiZ+fn3z55ZeWeWbOnCnBwcHy/fffy759++TOO++s9LLCjh07yvbt2+W3336TFi1aNKhLK8sbM2aMNG7c2HK56XfffSfh4eHy3HPPWebxhDbJycmRv/76S/766y8BIO+995789ddfliscHNEGmZmZEhUVJaNHj5YDBw7IsmXLxM/Pr95eWlldmxQVFckdd9whTZo0kT179lj93i1/hYe7tUlduEWwEBGZM2eONG3aVLRarXTt2lW2bdumdElOAaDSx6JFiyzzFBQUyGOPPSYhISHi5+cnd911l5w7d85qPSdPnpTBgweLr6+vhIeHy+TJk6W4uNjFe+M8FYOFJ7bJjz/+KG3bthWdTietWrWSjz76yOp1k8kkL7/8skRFRYlOp5MBAwZIcnKy1TyXL1+WkSNHSkBAgAQFBcm4ceMkJyfHlbvhMNnZ2TJx4kRp2rSp+Pj4SLNmzeSll16y+nDwhDbZtGlTpb9DxowZIyKOa4O9e/dKz549RafTSePGjWXmzJmu2kW7VdcmJ06cqPL37qZNmyzrcLc2qQuVSLlh54iIiIjqoMH3sSAiIqL6g8GCiIiIHIbBgoiIiByGwYKIiIgchsGCiIiIHIbBgoiIiByGwYKIiIgchsGCiIiIHIbBgoiIiByGwYKIiIgchsGCiIiIHOb/A8ivILzRL/4NAAAAAElFTkSuQmCC",
+      "text/plain": [
+       "<Figure size 640x480 with 1 Axes>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "plt.plot([sum(min(l, m) for l in lens) for m in range(max(lens) + 1)])\n",
+    "plt.title(\"Token coverage as a function of max length\")\n",
+    "plt.axvline(x=256, color=\"red\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "dl3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.8"
+  },
+  "orig_nbformat": 4,
+  "vscode": {
+   "interpreter": {
+    "hash": "90bfda469df5ac7fed8d7e225d563f60a7a7aa420ccfadb091c914debf775e49"
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

SVFT-main/LLM-Adapters/mathqa.py

@@ -0,0 +1,27 @@
+import os
+import json
+from datasets import load_dataset
+
+dataset = load_dataset("math_qa")
+save_path = "dataset/mathqa/test.json"
+
+if not os.path.exists("dataset/mathqa/"):
+    os.makedirs("dataset/mathqa/")
+
+
+def writer(data, save_path):
+    with open(save_path, "w") as f:
+        json.dump(data, f, indent=4)
+
+test_data = []
+for sample in dataset["test"]:
+    options = sample["options"].replace("a", "A").replace("b", "B").replace("c", "C").replace("d", "D").replace("e", "E").replace("f", "F")
+    test_data.append({
+        "instruction": f"{sample['Problem']} The options: {options}",
+        "input": "",
+        "output": "",
+        "answer": sample["correct"].upper(),
+    })
+
+writer(test_data, save_path)
+

SVFT-main/LLM-Adapters/multi_dataset_eval.py

@@ -0,0 +1,49 @@
+from concurrent.futures import ProcessPoolExecutor
+import queue
+import subprocess
+import os
+
+def evaluate(dataset, gpu):
+    print('*******dataset:', dataset)
+    model_name = "Pythia_2B_143000_SVFT_CR15K"
+    save_dir= "results/" + model_name
+    
+    if not os.path.exists(save_dir):
+        try:
+            os.makedirs(save_dir)
+        except:
+            pass
+
+    save_path = os.path.join(save_dir, dataset + ".txt")
+    command = f"CUDA_VISIBLE_DEVICES={gpu} python commonsense_evaluate_latest.py \
+               --model LLaMA-7B \
+               --adapter LoRA \
+               --dataset {dataset} \
+               --base_model './{model_name}' \
+               --batch_size 1| tee -a {save_path}"
+
+    result = subprocess.run(command, shell=True, text=True, capture_output=False)
+    print(f"Evaluation results for dataset {dataset} on GPU {gpu}:\n{result.stdout}")
+    return gpu
+
+
+datasets = ["boolq", "social_i_qa", "piqa", "ARC-Easy", "ARC-Challenge", "winogrande", "openbookqa", "hellaswag"]
+
+gpus = [0, 0, 0, 0]
+tasks_queue = queue.Queue()
+gpu_queue = queue.Queue()
+
+for gpu in gpus:
+    gpu_queue.put(gpu)
+for task in datasets:
+    tasks_queue.put(task)
+
+num_processes = min(len(datasets), len(gpus))  # number of processes to run in parallel
+
+with ProcessPoolExecutor(max_workers=num_processes) as executor:
+    futures = [executor.submit(evaluate, tasks_queue.get(), gpu_queue.get()) for i in range(num_processes)]
+    for future in futures:
+        gpu_id = future.result()
+        gpu_queue.put(gpu_id)
+        if tasks_queue.qsize() > 0:
+            futures.append(executor.submit(evaluate, tasks_queue.get(), gpu_queue.get()))

SVFT-main/LLM-Adapters/peft/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

SVFT-main/LLM-Adapters/peft/Makefile

@@ -0,0 +1,20 @@
+.PHONY: quality style test docs
+
+check_dirs := src tests examples
+
+# Check that source code meets quality standards
+
+# this target runs checks on all files
+quality:
+	black --check $(check_dirs)
+	ruff $(check_dirs)
+	doc-builder style src tests --max_len 119 --check_only
+
+# Format source code automatically and check is there are any problems left that need manual fixing
+style:
+	black $(check_dirs)
+	ruff $(check_dirs) --fix
+	doc-builder style src tests --max_len 119
+
+test:
+	pytest tests/

SVFT-main/LLM-Adapters/peft/pyproject.toml

@@ -0,0 +1,36 @@
+[tool.black]
+line-length = 119
+target-version = ['py36']
+
+[tool.ruff]
+ignore = ["C901", "E501", "E741", "W605"]
+select = ["C", "E", "F", "I", "W"]
+line-length = 119
+
+[tool.ruff.isort]
+lines-after-imports = 2
+known-first-party = ["peft"]
+
+[isort]
+default_section = "FIRSTPARTY"
+known_first_party = "peft"
+known_third_party = [
+    "numpy",
+    "torch",
+    "accelerate",
+    "transformers",
+]
+line_length = 119
+lines_after_imports = 2
+multi_line_output = 3
+include_trailing_comma = true
+force_grid_wrap = 0
+use_parentheses = true
+ensure_newline_before_comments = true
+
+[tool.pytest]
+doctest_optionflags = [
+    "NORMALIZE_WHITESPACE",
+    "ELLIPSIS",
+    "NUMBER",
+]

SVFT-main/LLM-Adapters/peft/setup.py

@@ -0,0 +1,76 @@
+# Copyright 2023 The HuggingFace Team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from setuptools import find_packages, setup
+
+extras = {}
+extras["quality"] = ["black ~= 22.0", "ruff>=0.0.241"]
+extras["docs_specific"] = ["hf-doc-builder"]
+extras["dev"] = extras["quality"] + extras["docs_specific"]
+extras["test"] = extras["dev"] + ["pytest", "pytest-xdist", "parameterized"]
+
+setup(
+    name="peft",
+    version="0.3.0.dev0",
+    description="Parameter-Efficient Fine-Tuning (PEFT)",
+    license_files=["LICENSE"],
+    keywords="deep learning",
+    license="Apache",
+    author="The AGI-Edgerunners team",
+    author_email="[email protected]",
+    url="https://github.com/AGI-Edgerunners/LLM-Adapters",
+    package_dir={"": "src"},
+    packages=find_packages("src"),
+    entry_points={},
+    python_requires=">=3.7.0",
+    install_requires=[
+        "numpy>=1.17",
+        "packaging>=20.0",
+        "psutil",
+        "pyyaml",
+        "torch>=1.13.0",
+        "transformers",
+        "accelerate",
+    ],
+    extras_require=extras,
+    classifiers=[
+        "Development Status :: 5 - Production/Stable",
+        "Intended Audience :: Developers",
+        "Intended Audience :: Education",
+        "Intended Audience :: Science/Research",
+        "License :: OSI Approved :: Apache Software License",
+        "Operating System :: OS Independent",
+        "Programming Language :: Python :: 3",
+        "Programming Language :: Python :: 3.7",
+        "Topic :: Scientific/Engineering :: Artificial Intelligence",
+    ],
+)
+
+# Release checklist
+# 1. Change the version in __init__.py and setup.py.
+# 2. Commit these changes with the message: "Release: VERSION"
+# 3. Add a tag in git to mark the release: "git tag VERSION -m 'Adds tag VERSION for pypi' "
+#    Push the tag to git: git push --tags origin main
+# 4. Run the following commands in the top-level directory:
+#      python setup.py bdist_wheel
+#      python setup.py sdist
+# 5. Upload the package to the pypi test server first:
+#      twine upload dist/* -r pypitest
+#      twine upload dist/* -r pypitest --repository-url=https://test.pypi.org/legacy/
+# 6. Check that you can install it in a virtualenv by running:
+#      pip install -i https://testpypi.python.org/pypi peft
+# 7. Upload the final version to actual pypi:
+#      twine upload dist/* -r pypi
+# 8. Add release notes to the tag in github once everything is looking hunky-dory.
+# 9. Update the version in __init__.py, setup.py to the new version "-dev" and push to master

SVFT-main/LLM-Adapters/peft/src/peft/__init__.py

@@ -0,0 +1,55 @@
+# flake8: noqa
+# There's no way to ignore "F401 '...' imported but unused" warnings in this
+# module, but to preserve other warnings. So, don't check this module at all.
+
+# coding=utf-8
+# Copyright 2023-present the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+__version__ = "0.3.0.dev0"
+
+from .mapping import MODEL_TYPE_TO_PEFT_MODEL_MAPPING, PEFT_TYPE_TO_CONFIG_MAPPING, get_peft_config, get_peft_model
+from .peft_model import (
+    PeftModel,
+    PeftModelForCausalLM,
+    PeftModelForSeq2SeqLM,
+    PeftModelForSequenceClassification,
+    PeftModelForTokenClassification,
+)
+from .tuners import (
+    LoraConfig,
+    LoraModel,
+    BottleneckConfig,
+    BottleneckModel,
+    PrefixEncoder,
+    PrefixTuningConfig,
+    PromptEmbedding,
+    PromptEncoder,
+    PromptEncoderConfig,
+    PromptEncoderReparameterizationType,
+    PromptTuningConfig,
+    PromptTuningInit,
+)
+from .utils import (
+    TRANSFORMERS_MODELS_TO_PREFIX_TUNING_POSTPROCESS_MAPPING,
+    PeftConfig,
+    PeftType,
+    PromptLearningConfig,
+    TaskType,
+    bloom_model_postprocess_past_key_value,
+    get_peft_model_state_dict,
+    prepare_model_for_int8_training,
+    set_peft_model_state_dict,
+    shift_tokens_right,
+)

SVFT-main/LLM-Adapters/peft/src/peft/mapping.py

@@ -0,0 +1,202 @@
+# coding=utf-8
+# Copyright 2023-present the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from .peft_model import (
+    PeftModel,
+    PeftModelForCausalLM,
+    PeftModelForSeq2SeqLM,
+    PeftModelForSequenceClassification,
+    PeftModelForTokenClassification,
+)
+from .tuners import LoraConfig, PrefixTuningConfig, PromptEncoderConfig, PromptTuningConfig, BottleneckConfig
+from .utils import PromptLearningConfig
+
+
+MODEL_TYPE_TO_PEFT_MODEL_MAPPING = {
+    "SEQ_CLS": PeftModelForSequenceClassification,
+    "SEQ_2_SEQ_LM": PeftModelForSeq2SeqLM,
+    "CAUSAL_LM": PeftModelForCausalLM,
+    "TOKEN_CLS": PeftModelForTokenClassification,
+}
+
+PEFT_TYPE_TO_CONFIG_MAPPING = {
+    "PROMPT_TUNING": PromptTuningConfig,
+    "PREFIX_TUNING": PrefixTuningConfig,
+    "P_TUNING": PromptEncoderConfig,
+    "LORA": LoraConfig,
+    "BOTTLENECK": BottleneckConfig,
+}
+
+TRANSFORMERS_MODELS_TO_LORA_TARGET_MODULES_MAPPING = {
+    "t5": ["q", "v"],
+    "mt5": ["q", "v"],
+    "bart": ["q_proj", "v_proj"],
+    "gpt2": ["c_attn"],
+    "bloom": ["query_key_value"],
+    "opt": ["q_proj", "v_proj"],
+    "gptj": ["q_proj", "v_proj"],
+    "gpt_neox": ["query_key_value"],
+    "gpt_neo": ["q_proj", "v_proj"],
+    "bert": ["query", "value"],
+    "roberta": ["query", "value"],
+    "xlm-roberta": ["query", "value"],
+    "electra": ["query", "value"],
+    "deberta-v2": ["query_proj", "value_proj"],
+    "deberta": ["in_proj"],
+    "layoutlm": ["query", "value"],
+    "llama": ["q_proj", "v_proj"],
+    "chatglm": ["query_key_value"],
+}
+
+TRANSFORMERS_MODELS_TO_BOTTLENECK_TARGET_MODULES_MAPPING = {
+    "bloom": ["dense_h_to_4h", "dense_4h_to_h"],
+    "gptj": ["fc_in", "fc_out"],
+    "gpt_neo": ["c_fc", "c_proj"],
+    "llama": ["gate_proj", "up_proj", "down_proj"],
+    "opt": ["fc1", "fc2"],
+    "chatglm": ["dense_h_to_4h", "dense_4h_to_h"],
+}
+
+TRANSFORMERS_MODELS_TO_ADAPTERP_TARGET_MODULES_MAPPING = {
+    "bloom": ["dense_4h_to_h"],
+    "gptj": ["fc_out"],
+    "gpt_neo": ["c_proj"],
+    "llama": ["down_proj"],
+    "opt": ["fc2"],
+    "chatglm": ["dense_4h_to_h"],
+}
+
+TRANSFORMERS_MODELS_TO_PARALLEL_TARGET_MODULES_MAPPING = {
+    "bloom": ["query_key_value"],
+    "gptj": ["q_proj", "v_proj", "k_proj"],
+    "gpt_neo": ["q_proj", "v_proj", "k_proj"],
+    "llama": ["q_proj", "v_proj", "k_proj"],
+    "opt": ["q_proj", "v_proj", "k_proj"],
+    "chatglm": ["query_key_value"],
+}
+
+
+
+def get_peft_config(config_dict):
+    """
+    Returns a Peft config object from a dictionary.
+
+    Args:
+        config_dict (`Dict[str, Any]`): Dictionary containing the configuration parameters.
+    """
+
+    return PEFT_TYPE_TO_CONFIG_MAPPING[config_dict["peft_type"]](**config_dict)
+
+
+def _prepare_prompt_learning_config(peft_config, model_config):
+    if peft_config.num_layers is None:
+        if "num_hidden_layers" in model_config:
+            num_layers = model_config["num_hidden_layers"]
+        elif "num_layers" in model_config:
+            num_layers = model_config["num_layers"]
+        elif "n_layer" in model_config:
+            num_layers = model_config["n_layer"]
+        else:
+            raise ValueError("Please specify `num_layers` in `peft_config`")
+        peft_config.num_layers = num_layers
+
+    if peft_config.token_dim is None:
+        if "hidden_size" in model_config:
+            token_dim = model_config["hidden_size"]
+        elif "n_embd" in model_config:
+            token_dim = model_config["n_embd"]
+        elif "d_model" in model_config:
+            token_dim = model_config["d_model"]
+        else:
+            raise ValueError("Please specify `token_dim` in `peft_config`")
+        peft_config.token_dim = token_dim
+
+    if peft_config.num_attention_heads is None:
+        if "num_attention_heads" in model_config:
+            num_attention_heads = model_config["num_attention_heads"]
+        elif "n_head" in model_config:
+            num_attention_heads = model_config["n_head"]
+        elif "num_heads" in model_config:
+            num_attention_heads = model_config["num_heads"]
+        elif "encoder_attention_heads" in model_config:
+            num_attention_heads = model_config["encoder_attention_heads"]
+        else:
+            raise ValueError("Please specify `num_attention_heads` in `peft_config`")
+        peft_config.num_attention_heads = num_attention_heads
+
+    if getattr(peft_config, "encoder_hidden_size", None) is None:
+        setattr(peft_config, "encoder_hidden_size", token_dim)
+
+    return peft_config
+
+
+def _prepare_lora_config(peft_config, model_config):
+    if peft_config.target_modules is None:
+        if model_config["model_type"] not in TRANSFORMERS_MODELS_TO_LORA_TARGET_MODULES_MAPPING:
+            raise ValueError("Please specify `target_modules` in `peft_config`")
+        peft_config.target_modules = TRANSFORMERS_MODELS_TO_LORA_TARGET_MODULES_MAPPING[model_config["model_type"]]
+    if len(peft_config.target_modules) == 1:
+        peft_config.fan_in_fan_out = True
+        peft_config.enable_lora = [True, False, True]
+    if peft_config.inference_mode:
+        peft_config.merge_weights = True
+    return peft_config
+
+
+def _prepare_bottleneck_config(peft_config, model_config):
+    if peft_config.target_modules is None:
+        if peft_config.use_parallel_adapter:
+            if model_config["model_type"] not in TRANSFORMERS_MODELS_TO_PARALLEL_TARGET_MODULES_MAPPING:
+                raise ValueError("Please specify `target_modules` in `peft_config`")
+            peft_config.target_modules = TRANSFORMERS_MODELS_TO_PARALLEL_TARGET_MODULES_MAPPING[model_config["model_type"]]
+        elif peft_config.use_adapterp:
+            if model_config["model_type"] not in TRANSFORMERS_MODELS_TO_ADAPTERP_TARGET_MODULES_MAPPING:
+                raise ValueError("Please specify `target_modules` in `peft_config`")
+            peft_config.target_modules = TRANSFORMERS_MODELS_TO_ADAPTERP_TARGET_MODULES_MAPPING[model_config["model_type"]]
+        else:
+            if model_config["model_type"] not in TRANSFORMERS_MODELS_TO_BOTTLENECK_TARGET_MODULES_MAPPING:
+                raise ValueError("Please specify `target_modules` in `peft_config`")
+            peft_config.target_modules = TRANSFORMERS_MODELS_TO_BOTTLENECK_TARGET_MODULES_MAPPING[model_config["model_type"]]
+
+    return peft_config
+    
+
+
+def get_peft_model(model, peft_config):
+    """
+    Returns a Peft model object from a model and a config.
+
+    Args:
+        model ([`transformers.PreTrainedModel`]): Model to be wrapped.
+        peft_config ([`PeftConfig`]): Configuration object containing the parameters of the Peft model.
+    """
+
+    model_config = model.config.to_dict()
+    peft_config.base_model_name_or_path = model.__dict__.get("name_or_path", None)
+    if peft_config.task_type not in MODEL_TYPE_TO_PEFT_MODEL_MAPPING.keys():
+        if peft_config.peft_type == "LORA":
+            peft_config = _prepare_lora_config(peft_config, model_config)
+            return PeftModel(model, peft_config)
+        elif peft_config.peft_type == "BOTTLENECK":
+            peft_config = _prepare_bottleneck_config(peft_config, model_config)
+            return PeftModel(model, peft_config)
+    if not isinstance(peft_config, PromptLearningConfig):
+        if peft_config.peft_type == "BOTTLENECK":
+            peft_config = _prepare_bottleneck_config(peft_config, model_config)
+        elif peft_config.peft_type == "LORA":
+            peft_config = _prepare_lora_config(peft_config, model_config)
+    else:
+        peft_config = _prepare_prompt_learning_config(peft_config, model_config)
+    return MODEL_TYPE_TO_PEFT_MODEL_MAPPING[peft_config.task_type](model, peft_config)

SVFT-main/LLM-Adapters/peft/src/peft/peft_model.py

@@ -0,0 +1,974 @@
+# coding=utf-8
+# Copyright 2023-present the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import inspect
+import os
+import warnings
+from contextlib import contextmanager
+
+import torch
+from accelerate import dispatch_model, infer_auto_device_map
+from accelerate.hooks import AlignDevicesHook, add_hook_to_module, remove_hook_from_submodules
+from accelerate.utils import get_balanced_memory
+from huggingface_hub import hf_hub_download
+from torch.nn import BCEWithLogitsLoss, CrossEntropyLoss, MSELoss
+from transformers import PreTrainedModel
+from transformers.modeling_outputs import SequenceClassifierOutput, TokenClassifierOutput
+from transformers.utils import PushToHubMixin
+
+from .tuners import LoraModel, BottleneckModel, PrefixEncoder, PromptEmbedding, PromptEncoder
+from .utils import (
+    TRANSFORMERS_MODELS_TO_PREFIX_TUNING_POSTPROCESS_MAPPING,
+    WEIGHTS_NAME,
+    PeftConfig,
+    PeftType,
+    PromptLearningConfig,
+    TaskType,
+    _set_trainable,
+    get_peft_model_state_dict,
+    set_peft_model_state_dict,
+    shift_tokens_right,
+)
+
+
+class PeftModel(PushToHubMixin, torch.nn.Module):
+    """
+    Parameter-Efficient Fine-Tuning Model. Base model encompassing various Peft methods.
+
+    Args:
+        model ([`PreTrainedModel`]): The base transformer model used for Peft.
+        peft_config ([`PeftConfig`]): The configuration of the Peft model.
+
+
+    **Attributes**:
+        - **base_model** ([`PreTrainedModel`]) -- The base transformer model used for Peft.
+        - **peft_config** ([`PeftConfig`]) -- The configuration of the Peft model.
+        - **modules_to_save** (`list` of `str`) -- The list of sub-module names to save when
+        saving the model.
+        - **prompt_encoder** ([`PromptEncoder`]) -- The prompt encoder used for Peft if
+        `isinstance(self.peft_config, PromptLearningConfig)`.
+        - **prompt_tokens** (`torch.Tensor`) -- The virtual prompt tokens used for Peft if
+        `isinstance(self.peft_config, PromptLearningConfig)`.
+        - **transformer_backbone_name** (`str`) -- The name of the transformer
+        backbone in the base model if `isinstance(self.peft_config, PromptLearningConfig)`.
+        - **word_embeddings** (`torch.nn.Embedding`) -- The word embeddings of the transformer backbone
+        in the base model if `isinstance(self.peft_config, PromptLearningConfig)`.
+    """
+
+    def __init__(self, model, peft_config: PeftConfig):
+        super().__init__()
+        self.peft_config = peft_config
+        self.base_model = model
+        self.config = self.base_model.config
+        self.modules_to_save = None
+        if isinstance(self.peft_config, PromptLearningConfig):
+            self._setup_prompt_encoder()
+        else:
+            if self.peft_config.peft_type == PeftType.LORA:
+                self.base_model = LoraModel(peft_config, model)
+            elif self.peft_config.peft_type == PeftType.BOTTLENECK:
+                self.base_model = BottleneckModel(peft_config, model)
+        if getattr(self.peft_config, "modules_to_save", None) is not None:
+            self.modules_to_save = self.peft_config.modules_to_save
+            _set_trainable(self)
+        self.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+        self.base_model_torch_dtype = getattr(model, "dtype", None)
+
+    def save_pretrained(self, save_directory, **kwargs):
+        r"""
+        Args:
+        This function saves the adapter model and the adapter configuration files to a directory, so that it can be
+        re-loaded using the `LoraModel.from_pretrained` class method, and also used by the `LoraModel.push_to_hub`
+        method.
+            save_directory (`str`):
+                Directory where the adapter model and configuration files will be saved (will be created if it does not
+                exist).
+            **kwargs:
+                Additional keyword arguments passed along to the `push_to_hub` method.
+        """
+        if os.path.isfile(save_directory):
+            raise ValueError(f"Provided path ({save_directory}) should be a directory, not a file")
+        os.makedirs(save_directory, exist_ok=True)
+
+        # save only the trainable weights
+        output_state_dict = get_peft_model_state_dict(self, kwargs.get("state_dict", None))
+        torch.save(output_state_dict, os.path.join(save_directory, WEIGHTS_NAME))
+
+        # save the config and change the inference mode to `True`
+        if self.peft_config.base_model_name_or_path is None:
+            self.peft_config.base_model_name_or_path = (
+                self.base_model.__dict__.get("name_or_path", None)
+                if isinstance(self.peft_config, PromptLearningConfig)
+                else self.base_model.model.__dict__.get("name_or_path", None)
+            )
+        inference_mode = self.peft_config.inference_mode
+        self.peft_config.inference_mode = True
+        self.peft_config.save_pretrained(save_directory)
+        self.peft_config.inference_mode = inference_mode
+
+    @classmethod
+    def from_pretrained(cls, model, model_id, **kwargs):
+        r"""
+        Args:
+        Instantiate a `LoraModel` from a pretrained Lora configuration and weights.
+            model (`transformers.PreTrainedModel`):
+                The model to be adapted. The model should be initialized with the `from_pretrained` method. from
+                `transformers` library.
+            model_id (`str`):
+                The name of the Lora configuration to use. Can be either:
+                    - A string, the `model id` of a Lora configuration hosted inside a model repo on
+                        huggingface Hub
+                    - A path to a directory containing a Lora configuration file saved using the
+                        `save_pretrained` method, e.g., ``./my_lora_config_directory/``.
+        """
+        from .mapping import MODEL_TYPE_TO_PEFT_MODEL_MAPPING, PEFT_TYPE_TO_CONFIG_MAPPING
+
+        # load the config
+        config = PEFT_TYPE_TO_CONFIG_MAPPING[PeftConfig.from_pretrained(model_id).peft_type].from_pretrained(model_id)
+
+        if getattr(model, "hf_device_map", None) is not None:
+            remove_hook_from_submodules(model)
+
+        if config.task_type not in MODEL_TYPE_TO_PEFT_MODEL_MAPPING.keys():
+            model = cls(model, config)
+        else:
+            model = MODEL_TYPE_TO_PEFT_MODEL_MAPPING[config.task_type](model, config)
+
+        # load weights if any
+        if os.path.exists(os.path.join(model_id, WEIGHTS_NAME)):
+            filename = os.path.join(model_id, WEIGHTS_NAME)
+        else:
+            try:
+                filename = hf_hub_download(model_id, WEIGHTS_NAME)
+            except:  # noqa
+                raise ValueError(
+                    f"Can't find weights for {model_id} in {model_id} or in the Hugging Face Hub. "
+                    f"Please check that the file {WEIGHTS_NAME} is present at {model_id}."
+                )
+
+        adapters_weights = torch.load(
+            filename, map_location=torch.device("cuda" if torch.cuda.is_available() else "cpu")
+        )
+        # load the weights into the model
+        model = set_peft_model_state_dict(model, adapters_weights)
+        if getattr(model, "hf_device_map", None) is not None:
+            device_map = kwargs.get("device_map", "auto")
+            max_memory = kwargs.get("max_memory", None)
+            no_split_module_classes = model._no_split_modules
+            if device_map != "sequential":
+                max_memory = get_balanced_memory(
+                    model,
+                    max_memory=max_memory,
+                    no_split_module_classes=no_split_module_classes,
+                    low_zero=(device_map == "balanced_low_0"),
+                )
+            if isinstance(device_map, str):
+                device_map = infer_auto_device_map(
+                    model, max_memory=max_memory, no_split_module_classes=no_split_module_classes
+                )
+            model = dispatch_model(model, device_map=device_map)
+            hook = AlignDevicesHook(io_same_device=True)
+            if model.peft_config.peft_type == PeftType.LORA or model.peft_config.peft_type == PeftType.BOTTLENECK:
+                add_hook_to_module(model.base_model.model, hook)
+            else:
+                remove_hook_from_submodules(model.prompt_encoder)
+                add_hook_to_module(model.base_model, hook)
+        return model
+
+    def _setup_prompt_encoder(self):
+        transformer_backbone = None
+        for name, module in self.base_model.named_children():
+            for param in module.parameters():
+                param.requires_grad = False
+            if isinstance(module, PreTrainedModel):
+                # Make sure to freeze Tranformers model
+                if transformer_backbone is None:
+                    transformer_backbone = module
+                    self.transformer_backbone_name = name
+
+        if self.peft_config.num_transformer_submodules is None:
+            self.peft_config.num_transformer_submodules = (
+                2 if self.peft_config.task_type == TaskType.SEQ_2_SEQ_LM else 1
+            )
+
+        for named_param, value in list(transformer_backbone.named_parameters()):
+            if value.shape[0] == self.base_model.config.vocab_size:
+                self.word_embeddings = transformer_backbone.get_submodule(named_param.replace(".weight", ""))
+                break
+
+        if self.peft_config.peft_type == PeftType.PROMPT_TUNING:
+            prompt_encoder = PromptEmbedding(self.peft_config, self.word_embeddings)
+        elif self.peft_config.peft_type == PeftType.P_TUNING:
+            prompt_encoder = PromptEncoder(self.peft_config)
+        elif self.peft_config.peft_type == PeftType.PREFIX_TUNING:
+            prompt_encoder = PrefixEncoder(self.peft_config)
+        else:
+            raise ValueError("Not supported")
+        self.prompt_encoder = prompt_encoder
+        self.prompt_tokens = torch.arange(
+            self.peft_config.num_virtual_tokens * self.peft_config.num_transformer_submodules
+        ).long()
+
+    def get_prompt_embedding_to_save(self):
+        """
+        Returns the prompt embedding to save when saving the model. Only applicable when `peft_config.peft_type !=
+        PeftType.LORA`.
+        """
+        prompt_tokens = self.prompt_tokens.unsqueeze(0).expand(1, -1).to(self.device)
+        if self.peft_config.peft_type == PeftType.PREFIX_TUNING:
+            prompt_tokens = prompt_tokens[:, : self.peft_config.num_virtual_tokens]
+        prompt_embeddings = self.prompt_encoder(prompt_tokens)
+        return prompt_embeddings[0].detach().cpu()
+
+    def get_prompt(self, batch_size):
+        """
+        Returns the virtual prompts to use for Peft. Only applicable when `peft_config.peft_type != PeftType.LORA`.
+        """
+        prompt_tokens = self.prompt_tokens.unsqueeze(0).expand(batch_size, -1).to(self.device)
+        if self.peft_config.peft_type == PeftType.PREFIX_TUNING:
+            prompt_tokens = prompt_tokens[:, : self.peft_config.num_virtual_tokens]
+            if self.peft_config.inference_mode:
+                past_key_values = self.prompt_encoder.embedding.weight.repeat(batch_size, 1, 1)
+            else:
+                past_key_values = self.prompt_encoder(prompt_tokens)
+            past_key_values = past_key_values.view(
+                batch_size,
+                self.peft_config.num_virtual_tokens,
+                self.peft_config.num_layers * 2,
+                self.peft_config.num_attention_heads,
+                self.peft_config.token_dim // self.peft_config.num_attention_heads,
+            )
+            if self.peft_config.num_transformer_submodules == 2:
+                past_key_values = torch.cat([past_key_values, past_key_values], dim=2)
+            past_key_values = past_key_values.permute([2, 0, 3, 1, 4]).split(
+                self.peft_config.num_transformer_submodules * 2
+            )
+            if TRANSFORMERS_MODELS_TO_PREFIX_TUNING_POSTPROCESS_MAPPING.get(self.config.model_type, None) is not None:
+                post_process_fn = TRANSFORMERS_MODELS_TO_PREFIX_TUNING_POSTPROCESS_MAPPING[self.config.model_type]
+                past_key_values = post_process_fn(past_key_values)
+            return past_key_values
+        else:
+            if self.peft_config.inference_mode:
+                prompts = self.prompt_encoder.embedding.weight.repeat(batch_size, 1, 1)
+            else:
+                prompts = self.prompt_encoder(prompt_tokens)
+            return prompts
+
+    def print_trainable_parameters(self):
+        """
+        Prints the number of trainable parameters in the model.
+        """
+        trainable_params = 0
+        all_param = 0
+        for _, param in self.named_parameters():
+            num_params = param.numel()
+            # if using DS Zero 3 and the weights are initialized empty
+            if num_params == 0 and hasattr(param, "ds_numel"):
+                num_params = param.ds_numel
+
+            all_param += num_params
+            if param.requires_grad:
+                trainable_params += num_params
+        print(
+            f"trainable params: {trainable_params} || all params: {all_param} || trainable%: {100 * trainable_params / all_param}"
+        )
+
+    def __getattr__(self, name: str):
+        """Forward missing attributes to the wrapped module."""
+        try:
+            return super().__getattr__(name)  # defer to nn.Module's logic
+        except AttributeError:
+            return getattr(self.base_model, name)
+
+    def forward(self, *args, **kwargs):
+        """
+        Forward pass of the model.
+        """
+        return self.get_base_model()(*args, **kwargs)
+
+    @contextmanager
+    def disable_adapter(self):
+        """
+        Disables the adapter module.
+        """
+        if isinstance(self.peft_config, PromptLearningConfig):
+            old_forward = self.forward
+            self.forward = self.base_model.forward
+        else:
+            self.base_model.disable_adapter_layers()
+        yield
+        if isinstance(self.peft_config, PromptLearningConfig):
+            self.forward = old_forward
+        else:
+            self.base_model.enable_adapter_layers()
+
+    def get_base_model(self):
+        """
+        Returns the base model.
+        """
+        return self.base_model if isinstance(self.peft_config, PromptLearningConfig) else self.base_model.model
+
+
+class PeftModelForSequenceClassification(PeftModel):
+    """
+    Peft model for sequence classification tasks.
+
+    Args:
+        model ([`PreTrainedModel`]): Base transformer model
+        peft_config ([`PeftConfig`]): Peft config.
+
+    **Attributes**:
+        - **config** ([`PretrainedConfig`]) -- The configuration object of the base model.
+        - **cls_layer_name** (`str`) -- The name of the classification layer.
+
+    Example::
+
+        >>> from transformers import AutoModelForSequenceClassification >>> from peft import
+        PeftModelForSequenceClassification, get_peft_config >>> config = {
+                'peft_type': 'PREFIX_TUNING', 'task_type': 'SEQ_CLS', 'inference_mode': False, 'num_virtual_tokens':
+                20, 'token_dim': 768, 'num_transformer_submodules': 1, 'num_attention_heads': 12, 'num_layers': 12,
+                'encoder_hidden_size': 768, 'prefix_projection': False, 'postprocess_past_key_value_function': None
+            }
+        >>> peft_config = get_peft_config(config) >>> model =
+        AutoModelForSequenceClassification.from_pretrained("bert-base-cased") >>> peft_model =
+        PeftModelForSequenceClassification(model, peft_config) >>> peft_model.print_trainable_parameters() trainable
+        params: 370178 || all params: 108680450 || trainable%: 0.3406113979101117
+    """
+
+    def __init__(self, model, peft_config: PeftConfig):
+        super().__init__(model, peft_config)
+        self.modules_to_save = ["classifier", "score"]
+
+        for name, _ in self.base_model.named_children():
+            if any(module_name in name for module_name in self.modules_to_save):
+                self.cls_layer_name = name
+                break
+
+        # to make sure classifier layer is trainable
+        _set_trainable(self)
+
+    def forward(
+        self,
+        input_ids=None,
+        attention_mask=None,
+        inputs_embeds=None,
+        labels=None,
+        output_attentions=None,
+        output_hidden_states=None,
+        return_dict=None,
+        **kwargs,
+    ):
+        return_dict = return_dict if return_dict is not None else self.config.use_return_dict
+
+        if not isinstance(self.peft_config, PromptLearningConfig):
+            return self.base_model(
+                input_ids=input_ids,
+                attention_mask=attention_mask,
+                inputs_embeds=inputs_embeds,
+                labels=labels,
+                output_attentions=output_attentions,
+                output_hidden_states=output_hidden_states,
+                return_dict=return_dict,
+                **kwargs,
+            )
+
+        batch_size = input_ids.shape[0]
+        if attention_mask is not None:
+            # concat prompt attention mask
+            prefix_attention_mask = torch.ones(batch_size, self.peft_config.num_virtual_tokens).to(self.device)
+            attention_mask = torch.cat((prefix_attention_mask, attention_mask), dim=1)
+        if kwargs.get("position_ids", None) is not None:
+            warnings.warn("Position ids are not supported for parameter efficient tuning. Ignoring position ids.")
+            kwargs["position_ids"] = None
+        kwargs.update(
+            {
+                "attention_mask": attention_mask,
+                "labels": labels,
+                "output_attentions": output_attentions,
+                "output_hidden_states": output_hidden_states,
+                "return_dict": return_dict,
+            }
+        )
+
+        if self.peft_config.peft_type == PeftType.PREFIX_TUNING:
+            return self._prefix_tuning_forward(input_ids=input_ids, **kwargs)
+        else:
+            if kwargs.get("token_type_ids", None) is not None:
+                kwargs["token_type_ids"] = torch.cat(
+                    (
+                        torch.zeros(batch_size, self.peft_config.num_virtual_tokens).to(self.device),
+                        kwargs["token_type_ids"],
+                    ),
+                    dim=1,
+                ).long()
+            if inputs_embeds is None:
+                inputs_embeds = self.word_embeddings(input_ids)
+            prompts = self.get_prompt(batch_size=batch_size)
+            prompts = prompts.to(inputs_embeds.dtype)
+            inputs_embeds = torch.cat((prompts, inputs_embeds), dim=1)
+            return self.base_model(inputs_embeds=inputs_embeds, **kwargs)
+
+    def _prefix_tuning_forward(
+        self,
+        input_ids=None,
+        attention_mask=None,
+        inputs_embeds=None,
+        labels=None,
+        output_attentions=None,
+        output_hidden_states=None,
+        return_dict=None,
+        **kwargs,
+    ):
+        batch_size = input_ids.shape[0]
+        past_key_values = self.get_prompt(batch_size)
+        fwd_params = list(inspect.signature(self.base_model.forward).parameters.keys())
+        kwargs.update(
+            {
+                "input_ids": input_ids,
+                "attention_mask": attention_mask,
+                "inputs_embeds": inputs_embeds,
+                "output_attentions": output_attentions,
+                "output_hidden_states": output_hidden_states,
+                "return_dict": return_dict,
+                "past_key_values": past_key_values,
+            }
+        )
+        if "past_key_values" in fwd_params:
+            return self.base_model(labels=labels, **kwargs)
+        else:
+            transformer_backbone_name = self.base_model.get_submodule(self.transformer_backbone_name)
+            fwd_params = list(inspect.signature(transformer_backbone_name.forward).parameters.keys())
+            if "past_key_values" not in fwd_params:
+                raise ValueError("Model does not support past key values which are required for prefix tuning.")
+            outputs = transformer_backbone_name(**kwargs)
+            pooled_output = outputs[1] if len(outputs) > 1 else outputs[0]
+            if "dropout" in [name for name, _ in list(self.base_model.named_children())]:
+                pooled_output = self.base_model.dropout(pooled_output)
+            logits = self.base_model.get_submodule(self.cls_layer_name)(pooled_output)
+
+            loss = None
+            if labels is not None:
+                if self.config.problem_type is None:
+                    if self.base_model.num_labels == 1:
+                        self.config.problem_type = "regression"
+                    elif self.base_model.num_labels > 1 and (labels.dtype == torch.long or labels.dtype == torch.int):
+                        self.config.problem_type = "single_label_classification"
+                    else:
+                        self.config.problem_type = "multi_label_classification"
+
+                if self.config.problem_type == "regression":
+                    loss_fct = MSELoss()
+                    if self.base_model.num_labels == 1:
+                        loss = loss_fct(logits.squeeze(), labels.squeeze())
+                    else:
+                        loss = loss_fct(logits, labels)
+                elif self.config.problem_type == "single_label_classification":
+                    loss_fct = CrossEntropyLoss()
+                    loss = loss_fct(logits.view(-1, self.base_model.num_labels), labels.view(-1))
+                elif self.config.problem_type == "multi_label_classification":
+                    loss_fct = BCEWithLogitsLoss()
+                    loss = loss_fct(logits, labels)
+            if not return_dict:
+                output = (logits,) + outputs[2:]
+                return ((loss,) + output) if loss is not None else output
+
+            return SequenceClassifierOutput(
+                loss=loss,
+                logits=logits,
+                hidden_states=outputs.hidden_states,
+                attentions=outputs.attentions,
+            )
+
+
+class PeftModelForCausalLM(PeftModel):
+    """
+    Peft model for Causal LM
+
+    Args:
+        model ([`PreTrainedModel`]): Base transformer model
+        peft_config ([`PeftConfig`]): Peft config.
+
+
+    Example::
+
+        >>> from transformers import AutoModelForCausalLM >>> from peft import PeftModelForCausalLM, get_peft_config
+        >>> config = {
+                'peft_type': 'PREFIX_TUNING', 'task_type': 'CAUSAL_LM', 'inference_mode': False, 'num_virtual_tokens':
+                20, 'token_dim': 1280, 'num_transformer_submodules': 1, 'num_attention_heads': 20, 'num_layers': 36,
+                'encoder_hidden_size': 1280, 'prefix_projection': False, 'postprocess_past_key_value_function': None
+            }
+        >>> peft_config = get_peft_config(config) >>> model = AutoModelForCausalLM.from_pretrained("gpt2-large") >>>
+        peft_model = PeftModelForCausalLM(model, peft_config) >>> peft_model.print_trainable_parameters() trainable
+        params: 1843200 || all params: 775873280 || trainable%: 0.23756456724479544
+    """
+
+    def __init__(self, model, peft_config: PeftConfig):
+        super().__init__(model, peft_config)
+        self.base_model_prepare_inputs_for_generation = self.base_model.prepare_inputs_for_generation
+
+    def forward(
+        self,
+        input_ids=None,
+        attention_mask=None,
+        inputs_embeds=None,
+        labels=None,
+        output_attentions=None,
+        output_hidden_states=None,
+        return_dict=None,
+        **kwargs,
+    ):
+        if not isinstance(self.peft_config, PromptLearningConfig):
+            return self.base_model(
+                input_ids=input_ids,
+                attention_mask=attention_mask,
+                inputs_embeds=inputs_embeds,
+                labels=labels,
+                output_attentions=output_attentions,
+                output_hidden_states=output_hidden_states,
+                return_dict=return_dict,
+                **kwargs,
+            )
+
+        batch_size = input_ids.shape[0]
+        if attention_mask is not None:
+            # concat prompt attention mask
+            prefix_attention_mask = torch.ones(batch_size, self.peft_config.num_virtual_tokens).to(self.device)
+            attention_mask = torch.cat((prefix_attention_mask, attention_mask), dim=1)
+
+        if kwargs.get("position_ids", None) is not None:
+            warnings.warn("Position ids are not supported for parameter efficient tuning. Ignoring position ids.")
+            kwargs["position_ids"] = None
+        if kwargs.get("token_type_ids", None) is not None:
+            warnings.warn("Token type ids are not supported for parameter efficient tuning. Ignoring token type ids")
+            kwargs["token_type_ids"] = None
+        kwargs.update(
+            {
+                "attention_mask": attention_mask,
+                "labels": labels,
+                "output_attentions": output_attentions,
+                "output_hidden_states": output_hidden_states,
+                "return_dict": return_dict,
+            }
+        )
+
+        if self.peft_config.peft_type == PeftType.PREFIX_TUNING:
+            past_key_values = self.get_prompt(batch_size)
+            return self.base_model(input_ids=input_ids, past_key_values=past_key_values, **kwargs)
+        else:
+            if inputs_embeds is None:
+                inputs_embeds = self.word_embeddings(input_ids)
+            # concat prompt labels
+            if labels is not None:
+                prefix_labels = torch.full((batch_size, self.peft_config.num_virtual_tokens), -100).to(self.device)
+                kwargs["labels"] = torch.cat((prefix_labels, labels), dim=1)
+            prompts = self.get_prompt(batch_size=batch_size)
+            prompts = prompts.to(inputs_embeds.dtype)
+            inputs_embeds = torch.cat((prompts, inputs_embeds), dim=1)
+            return self.base_model(inputs_embeds=inputs_embeds, **kwargs)
+
+    def generate(self, **kwargs):
+        self.base_model.prepare_inputs_for_generation = self.prepare_inputs_for_generation
+        try:
+            if not isinstance(self.peft_config, PromptLearningConfig):
+                outputs = self.base_model.generate(**kwargs)
+            else:
+                if "input_ids" not in kwargs:
+                    raise ValueError("input_ids must be provided for Peft model generation")
+                if kwargs.get("attention_mask", None) is not None:
+                    # concat prompt attention mask
+                    prefix_attention_mask = torch.ones(
+                        kwargs["input_ids"].shape[0], self.peft_config.num_virtual_tokens
+                    ).to(kwargs["input_ids"].device)
+                    kwargs["attention_mask"] = torch.cat((prefix_attention_mask, kwargs["attention_mask"]), dim=1)
+
+                if kwargs.get("position_ids", None) is not None:
+                    warnings.warn(
+                        "Position ids are not supported for parameter efficient tuning. Ignoring position ids."
+                    )
+                    kwargs["position_ids"] = None
+                if kwargs.get("token_type_ids", None) is not None:
+                    warnings.warn(
+                        "Token type ids are not supported for parameter efficient tuning. Ignoring token type ids"
+                    )
+                    kwargs["token_type_ids"] = None
+
+                outputs = self.base_model.generate(**kwargs)
+        except:
+            self.base_model.prepare_inputs_for_generation = self.base_model_prepare_inputs_for_generation
+            raise
+        else:
+            self.base_model.prepare_inputs_for_generation = self.base_model_prepare_inputs_for_generation
+            return outputs
+
+    def prepare_inputs_for_generation(self, *args, **kwargs):
+        model_kwargs = self.base_model_prepare_inputs_for_generation(*args, **kwargs)
+        if isinstance(self.peft_config, PromptLearningConfig):
+            if self.peft_config.peft_type == PeftType.PREFIX_TUNING:
+                prefix_attention_mask = torch.ones(
+                    model_kwargs["input_ids"].shape[0], self.peft_config.num_virtual_tokens
+                ).to(model_kwargs["input_ids"].device)
+                model_kwargs["attention_mask"] = torch.cat(
+                    (prefix_attention_mask, model_kwargs["attention_mask"]), dim=1
+                )
+
+            if model_kwargs["past_key_values"] is None and self.peft_config.peft_type == PeftType.PREFIX_TUNING:
+                past_key_values = self.get_prompt(batch_size=model_kwargs["input_ids"].shape[0])
+                if self.base_model_torch_dtype is not None:
+                    # handle the case for Bloom where it outputs tuple of tuples
+                    if isinstance(past_key_values[0], tuple):
+                        past_key_values = tuple(
+                            tuple(
+                                past_key_value.to(self.base_model_torch_dtype)
+                                for past_key_value in past_key_value_tuple
+                            )
+                            for past_key_value_tuple in past_key_values
+                        )
+                    else:
+                        past_key_values = tuple(
+                            past_key_value.to(self.base_model_torch_dtype) for past_key_value in past_key_values
+                        )
+                        
+                model_kwargs["past_key_values"] = past_key_values
+            else:
+                if model_kwargs["past_key_values"] is None:
+                    inputs_embeds = self.word_embeddings(model_kwargs["input_ids"])
+                    prompts = self.get_prompt(batch_size=model_kwargs["input_ids"].shape[0])
+                    prompts = prompts.to(inputs_embeds.dtype)
+                    model_kwargs["inputs_embeds"] = torch.cat((prompts, inputs_embeds), dim=1)
+                    model_kwargs["input_ids"] = None
+
+        return model_kwargs
+
+
+class PeftModelForSeq2SeqLM(PeftModel):
+    """
+    Peft model for Seq2Seq LM
+
+    Args:
+        model ([`PreTrainedModel`]): Base transformer model
+        peft_config ([`PeftConfig`]): Peft config.
+
+
+    Example::
+
+        >>> from transformers import AutoModelForSeq2SeqLM >>> from peft import PeftModelForSeq2SeqLM, get_peft_config
+        >>> config = {
+                'peft_type': 'LORA', 'task_type': 'SEQ_2_SEQ_LM', 'inference_mode': False, 'r': 8, 'target_modules':
+                ['q', 'v'], 'lora_alpha': 32, 'lora_dropout': 0.1, 'merge_weights': False, 'fan_in_fan_out': False,
+                'enable_lora': None, 'bias': 'none'
+            }
+        >>> peft_config = get_peft_config(config) >>> model = AutoModelForSeq2SeqLM.from_pretrained("t5-base") >>>
+        peft_model = PeftModelForSeq2SeqLM(model, peft_config) >>> peft_model.print_trainable_parameters() trainable
+        params: 884736 || all params: 223843584 || trainable%: 0.3952474242013566
+    """
+
+    def __init__(self, model, peft_config: PeftConfig):
+        super().__init__(model, peft_config)
+        self.base_model_prepare_inputs_for_generation = self.base_model.prepare_inputs_for_generation
+        self.base_model_prepare_encoder_decoder_kwargs_for_generation = (
+            self.base_model._prepare_encoder_decoder_kwargs_for_generation
+        )
+
+    def forward(
+        self,
+        input_ids=None,
+        attention_mask=None,
+        inputs_embeds=None,
+        decoder_input_ids=None,
+        decoder_attention_mask=None,
+        decoder_inputs_embeds=None,
+        labels=None,
+        output_attentions=None,
+        output_hidden_states=None,
+        return_dict=None,
+        **kwargs,
+    ):
+        if not isinstance(self.peft_config, PromptLearningConfig):
+            return self.base_model(
+                input_ids=input_ids,
+                attention_mask=attention_mask,
+                inputs_embeds=inputs_embeds,
+                decoder_input_ids=decoder_input_ids,
+                decoder_attention_mask=decoder_attention_mask,
+                decoder_inputs_embeds=decoder_inputs_embeds,
+                labels=labels,
+                output_attentions=output_attentions,
+                output_hidden_states=output_hidden_states,
+                return_dict=return_dict,
+                **kwargs,
+            )
+
+        batch_size = input_ids.shape[0]
+        if decoder_attention_mask is not None:
+            # concat prompt attention mask
+            prefix_attention_mask = torch.ones(batch_size, self.peft_config.num_virtual_tokens).to(self.device)
+            decoder_attention_mask = torch.cat((prefix_attention_mask, decoder_attention_mask), dim=1)
+
+        if kwargs.get("position_ids", None) is not None:
+            warnings.warn("Position ids are not supported for parameter efficient tuning. Ignoring position ids.")
+            kwargs["position_ids"] = None
+        if kwargs.get("token_type_ids", None) is not None:
+            warnings.warn("Token type ids are not supported for parameter efficient tuning. Ignoring token type ids")
+            kwargs["token_type_ids"] = None
+        kwargs.update(
+            {
+                "attention_mask": attention_mask,
+                "decoder_attention_mask": decoder_attention_mask,
+                "labels": labels,
+                "output_attentions": output_attentions,
+                "output_hidden_states": output_hidden_states,
+                "return_dict": return_dict,
+            }
+        )
+
+        if self.peft_config.peft_type == PeftType.PREFIX_TUNING:
+            past_key_values = self.get_prompt(batch_size)
+            return self.base_model(
+                input_ids=input_ids, decoder_input_ids=decoder_input_ids, past_key_values=past_key_values, **kwargs
+            )
+        else:
+            if inputs_embeds is None:
+                inputs_embeds = self.word_embeddings(input_ids)
+            if decoder_inputs_embeds is None and decoder_input_ids is None:
+                decoder_input_ids = shift_tokens_right(
+                    labels, self.config.pad_token_id, self.config.decoder_start_token_id
+                )
+                decoder_inputs_embeds = self.word_embeddings(decoder_input_ids)
+
+            if attention_mask is not None:
+                # concat prompt attention mask
+                prefix_attention_mask = torch.ones(batch_size, self.peft_config.num_virtual_tokens).to(self.device)
+                kwargs["attention_mask"] = torch.cat((prefix_attention_mask, attention_mask), dim=1)
+            # concat prompt labels
+            if labels is not None:
+                if self.peft_config.num_transformer_submodules == 1:
+                    kwargs["labels"] = labels
+                elif self.peft_config.num_transformer_submodules == 2:
+                    prefix_labels = torch.full((batch_size, self.peft_config.num_virtual_tokens), -100).to(self.device)
+                    kwargs["labels"] = torch.cat((prefix_labels, labels), dim=1)
+            prompts = self.get_prompt(batch_size=batch_size)
+            prompts = prompts.to(inputs_embeds.dtype)
+            inputs_embeds = torch.cat((prompts[:, : self.peft_config.num_virtual_tokens], inputs_embeds), dim=1)
+            if self.peft_config.num_transformer_submodules == 1:
+                return self.base_model(inputs_embeds=inputs_embeds, **kwargs)
+            elif self.peft_config.num_transformer_submodules == 2:
+                decoder_inputs_embeds = torch.cat(
+                    (prompts[:, self.peft_config.num_virtual_tokens :], decoder_inputs_embeds), dim=1
+                )
+                return self.base_model(
+                    inputs_embeds=inputs_embeds, decoder_inputs_embeds=decoder_inputs_embeds, **kwargs
+                )
+
+    def generate(self, **kwargs):
+        self.base_model.prepare_inputs_for_generation = self.prepare_inputs_for_generation
+        self.base_model._prepare_encoder_decoder_kwargs_for_generation = (
+            self._prepare_encoder_decoder_kwargs_for_generation
+        )
+        try:
+            if not isinstance(self.peft_config, PromptLearningConfig):
+                outputs = self.base_model.generate(**kwargs)
+            else:
+                if "input_ids" not in kwargs:
+                    raise ValueError("input_ids must be provided for Peft model generation")
+                if kwargs.get("position_ids", None) is not None:
+                    warnings.warn(
+                        "Position ids are not supported for parameter efficient tuning. Ignoring position ids."
+                    )
+                    kwargs["position_ids"] = None
+                if kwargs.get("token_type_ids", None) is not None:
+                    warnings.warn(
+                        "Token type ids are not supported for parameter efficient tuning. Ignoring token type ids"
+                    )
+                    kwargs["token_type_ids"] = None
+
+                if self.peft_config.peft_type == PeftType.PREFIX_TUNING:
+                    outputs = self.base_model.generate(**kwargs)
+                else:
+                    raise NotImplementedError
+        except:
+            self.base_model.prepare_inputs_for_generation = self.base_model_prepare_inputs_for_generation
+            self.base_model._prepare_encoder_decoder_kwargs_for_generation = (
+                self.base_model_prepare_encoder_decoder_kwargs_for_generation
+            )
+            raise
+        else:
+            self.base_model.prepare_inputs_for_generation = self.base_model_prepare_inputs_for_generation
+            self.base_model._prepare_encoder_decoder_kwargs_for_generation = (
+                self.base_model_prepare_encoder_decoder_kwargs_for_generation
+            )
+            return outputs
+
+    def prepare_inputs_for_generation(self, *args, **kwargs):
+        model_kwargs = self.base_model_prepare_inputs_for_generation(*args, **kwargs)
+        if model_kwargs["past_key_values"] is None and self.peft_config.peft_type == PeftType.PREFIX_TUNING:
+            batch_size = model_kwargs["decoder_input_ids"].shape[0]
+            past_key_values = self.get_prompt(batch_size)
+            model_kwargs["past_key_values"] = past_key_values
+        return model_kwargs
+
+
+class PeftModelForTokenClassification(PeftModel):
+    """
+    Peft model for sequence classification tasks.
+
+    Args:
+        model ([`PreTrainedModel`]): Base transformer model
+        peft_config ([`PeftConfig`]): Peft config.
+
+    **Attributes**:
+        - **config** ([`PretrainedConfig`]) -- The configuration object of the base model.
+        - **cls_layer_name** (`str`) -- The name of the classification layer.
+
+    Example::
+
+        >>> from transformers import AutoModelForSequenceClassification >>> from peft import
+        PeftModelForTokenClassification, get_peft_config >>> config = {
+                'peft_type': 'PREFIX_TUNING', 'task_type': 'TOKEN_CLS', 'inference_mode': False, 'num_virtual_tokens':
+                20, 'token_dim': 768, 'num_transformer_submodules': 1, 'num_attention_heads': 12, 'num_layers': 12,
+                'encoder_hidden_size': 768, 'prefix_projection': False, 'postprocess_past_key_value_function': None
+            }
+        >>> peft_config = get_peft_config(config) >>> model =
+        AutoModelForTokenClassification.from_pretrained("bert-base-cased") >>> peft_model =
+        PeftModelForTokenClassification(model, peft_config) >>> peft_model.print_trainable_parameters() trainable
+        params: 370178 || all params: 108680450 || trainable%: 0.3406113979101117
+    """
+
+    def __init__(self, model, peft_config: PeftConfig):
+        super().__init__(model, peft_config)
+        self.modules_to_save = ["classifier", "score"]
+
+        for name, _ in self.base_model.named_children():
+            if any(module_name in name for module_name in self.modules_to_save):
+                self.cls_layer_name = name
+                break
+
+        # to make sure classifier layer is trainable
+        _set_trainable(self)
+
+    def forward(
+        self,
+        input_ids=None,
+        attention_mask=None,
+        inputs_embeds=None,
+        labels=None,
+        output_attentions=None,
+        output_hidden_states=None,
+        return_dict=None,
+        **kwargs,
+    ):
+        return_dict = return_dict if return_dict is not None else self.config.use_return_dict
+
+        if not isinstance(self.peft_config, PromptLearningConfig):
+            return self.base_model(
+                input_ids=input_ids,
+                attention_mask=attention_mask,
+                inputs_embeds=inputs_embeds,
+                labels=labels,
+                output_attentions=output_attentions,
+                output_hidden_states=output_hidden_states,
+                return_dict=return_dict,
+                **kwargs,
+            )
+
+        batch_size = input_ids.shape[0]
+        if attention_mask is not None:
+            # concat prompt attention mask
+            prefix_attention_mask = torch.ones(batch_size, self.peft_config.num_virtual_tokens).to(self.device)
+            attention_mask = torch.cat((prefix_attention_mask, attention_mask), dim=1)
+        if kwargs.get("position_ids", None) is not None:
+            warnings.warn("Position ids are not supported for parameter efficient tuning. Ignoring position ids.")
+            kwargs["position_ids"] = None
+        kwargs.update(
+            {
+                "attention_mask": attention_mask,
+                "labels": labels,
+                "output_attentions": output_attentions,
+                "output_hidden_states": output_hidden_states,
+                "return_dict": return_dict,
+            }
+        )
+
+        if self.peft_config.peft_type == PeftType.PREFIX_TUNING:
+            return self._prefix_tuning_forward(input_ids=input_ids, **kwargs)
+        else:
+            if kwargs.get("token_type_ids", None) is not None:
+                kwargs["token_type_ids"] = torch.cat(
+                    (
+                        torch.zeros(batch_size, self.peft_config.num_virtual_tokens).to(self.device),
+                        kwargs["token_type_ids"],
+                    ),
+                    dim=1,
+                ).long()
+            if inputs_embeds is None:
+                inputs_embeds = self.word_embeddings(input_ids)
+            prompts = self.get_prompt(batch_size=batch_size)
+            prompts = prompts.to(inputs_embeds.dtype)
+            inputs_embeds = torch.cat((prompts, inputs_embeds), dim=1)
+            return self.base_model(inputs_embeds=inputs_embeds, **kwargs)
+
+    def _prefix_tuning_forward(
+        self,
+        input_ids=None,
+        attention_mask=None,
+        inputs_embeds=None,
+        labels=None,
+        output_attentions=None,
+        output_hidden_states=None,
+        return_dict=None,
+        **kwargs,
+    ):
+        batch_size = input_ids.shape[0]
+        past_key_values = self.get_prompt(batch_size)
+        fwd_params = list(inspect.signature(self.base_model.forward).parameters.keys())
+        kwargs.update(
+            {
+                "input_ids": input_ids,
+                "attention_mask": attention_mask,
+                "inputs_embeds": inputs_embeds,
+                "output_attentions": output_attentions,
+                "output_hidden_states": output_hidden_states,
+                "return_dict": return_dict,
+                "past_key_values": past_key_values,
+            }
+        )
+        if "past_key_values" in fwd_params:
+            return self.base_model(labels=labels, **kwargs)
+        else:
+            transformer_backbone_name = self.base_model.get_submodule(self.transformer_backbone_name)
+            fwd_params = list(inspect.signature(transformer_backbone_name.forward).parameters.keys())
+            if "past_key_values" not in fwd_params:
+                raise ValueError("Model does not support past key values which are required for prefix tuning.")
+            outputs = transformer_backbone_name(**kwargs)
+            sequence_output = outputs[0]
+            if "dropout" in [name for name, _ in list(self.base_model.named_children())]:
+                sequence_output = self.base_model.dropout(sequence_output)
+            logits = self.base_model.get_submodule(self.cls_layer_name)(sequence_output)
+
+            loss = None
+            loss = None
+            if labels is not None:
+                loss_fct = CrossEntropyLoss()
+                loss = loss_fct(logits.view(-1, self.num_labels), labels.view(-1))
+
+            if not return_dict:
+                output = (logits,) + outputs[2:]
+                return ((loss,) + output) if loss is not None else output
+
+            return TokenClassifierOutput(
+                loss=loss,
+                logits=logits,
+                hidden_states=outputs.hidden_states,
+                attentions=outputs.attentions,
+            )

SVFT-main/LLM-Adapters/peft/src/peft/tuners/__init__.py

@@ -0,0 +1,24 @@
+# flake8: noqa
+# There's no way to ignore "F401 '...' imported but unused" warnings in this
+# module, but to preserve other warnings. So, don't check this module at all
+
+# coding=utf-8
+# Copyright 2023-present the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from .lora import LoraConfig, LoraModel
+from .bottleneck import BottleneckConfig, BottleneckModel
+from .p_tuning import PromptEncoder, PromptEncoderConfig, PromptEncoderReparameterizationType
+from .prefix_tuning import PrefixEncoder, PrefixTuningConfig
+from .prompt_tuning import PromptEmbedding, PromptTuningConfig, PromptTuningInit

SVFT-main/LLM-Adapters/peft/src/peft/tuners/bottleneck.py

@@ -0,0 +1,532 @@
+import importlib
+import math
+import re
+import warnings
+from dataclasses import asdict, dataclass, field
+from enum import Enum
+from typing import List, Optional, Union
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+from ..utils import PeftConfig, PeftType, transpose
+from transformers.activations import ACT2FN
+
+
+TRANSFORMERS_MODELS_TO_ADAPTER_TYPE_MAPPING = {
+    "bloom": {"dense_h_to_4h": "mh_adapter", "dense_4h_to_h": "output_adapter"},
+    "gptj": {"fc_in":"mh_adapter", "fc_out":"output_adapter"},
+    "gpt_neo": {"c_fc":"mh_adapter", "c_proj":"output_adapter"},
+    "llama": {"gate_proj": "mh_adapter", "up_proj":"mh_adapter", "down_proj":"output_adapter"},
+    "opt": {"fc1":"mh_adapter", "fc2":"output_adapter"},
+    "chatglm": {"dense_h_to_4h": "mh_adapter", "dense_4h_to_h": "output_adapter"},
+}
+
+def is_bnb_available():
+    return importlib.util.find_spec("bitsandbytes") is not None
+
+
+if is_bnb_available():
+    import bitsandbytes as bnb
+
+@dataclass
+class BottleneckConfig(PeftConfig):
+    """
+    This is the configuration class to store the configuration of a [`~peft.Bottleneck`].
+
+    Args:
+        bottleneck_size (`int`): The size of the bottleneck.
+        non_linearity (`str`): The non-linearity to apply to the bottleneck.
+        dropout (`float`, optional): The dropout probability of the bottleneck. Default to 0.0
+        bias ('str'): Bias type for Bottleneck. Can be 'none', 'all' or 'adapter_only'. Default to 'none'.
+        use_parallel_adapter (:obj:`bool`, optional): Whether to use parallel adapter. Defaults to False.
+        scaling (:obj:`float` or :obj:`str`, optional):
+            Scaling factor to use for scaled addition of adapter outputs as done by He et al. (2021). Can be either a
+            constant factor (float) or the string "learned", in which case the scaling factor is learned. Defaults to
+            1.0.
+        target_modules (`Union[List[str],str]`): The names of the modules to apply Adapter to.
+        init_weights (:obj:`str`, optional): Initialization method for the weights of the adapter modules.
+            Currently, this can be either "bert" (default) or "mam_adapter".
+        modules_to_save (`List[str]`):List of modules apart from Bottleneck adapter layers to be set as trainable
+            and saved in the final checkpoint.
+    """
+    bottleneck_size : int = field(default=256, metadata={"help": "The size of the bottleneck"})
+    non_linearity : str = field(default="tanh", metadata={"help": "The non-linearity to apply to the bottleneck"})
+    adapter_dropout : float = field(default=0.0, metadata={"help": "The dropout probability of the bottleneck, default to 0.0"})
+    target_modules: Optional[Union[List[str], str]] = field(
+        default=None,
+        metadata={
+            "help": "List of module names or regex expression of the module names to replace with Adapter."
+            "For example, ['q', 'v'] or '.*decoder.*(SelfAttention|EncDecAttention).*(q|v)$' "
+        },
+    )
+    use_parallel_adapter: bool = field(default=False, metadata={"help": "Whether to use parallel adapter"})
+    use_adapterp: bool = field(default=False, metadata={"help": "Whether to use adapterp"})
+    scaling: Union[float, str] = 1.0
+    bias: str = field(default="none", metadata={"help": "Bias type for Bottleneck. Can be 'none', 'all' or 'adapter_only'"})
+    init_weights: str = field(default="bert", metadata={"help": "Initialization method for the weights of the adapter modules."})
+    modules_to_save: Optional[List[str]] = field(
+        default=None,
+        metadata={
+            "help": "List of modules apart from Adapter layers to be set as trainable and saved in the final checkpoint. "
+            "For example, in Sequence Classification or Token Classification tasks, "
+            "the final layer `classifier/score` are randomly initialized and as such need to be trainable and saved."
+        },
+    )
+
+    def __post_init__(self):
+        self.peft_type = PeftType.BOTTLENECK
+
+
+class BottleneckModel(torch.nn.Module):
+    """
+    Creates Bottleneck adapter model for a pretrained trainsformers model.
+
+    Args:
+        model ('transformers.PreTrainedModel'): The pretrained model to be adapted.
+        config (`BottleneckConfig`): The configuration of the Bottleneck adapter.
+    
+    Returns:
+        `torch.nn.Module`: The Bottleneck adapter model.
+    
+    Example::
+
+        >>> from transformers import AutoModelForCausalLM, BottleneckConfig
+        >>> from peft import BottleneckModel, BottleneckConfig
+        >>> config = BottleneckConfig(
+            peft_type="BOTTLNECK", task="CAUSAL_LM", target_modules=["gate_proj", "up_proj", "down_proj"],
+            bottleneck_size=256, non_linearity="tanh",
+        )
+        >>> model = AutoModelForCausalLM.from_pretrained("decapoda-research/llama-7b-hf") 
+        >>> bottleneck_model = BottleneckModel(config, model)
+
+    **Attribute**:
+        - **model** (`transformers.PreTrainedModel`): The pretrained model to be adapted.
+        - **peft_config** (`BottleneckConfig`): The configuration of the Bottleneck adapter.
+    """
+
+    def __init__(self, config, model):
+        super().__init__()
+        self.model = model
+        self.peft_config = config
+        self._find_and_replace()
+        mark_only_adapter_as_trainable(self.model, self.peft_config.bias)
+        self.forward = self.model.forward
+
+    def _find_and_replace(self):
+        loaded_in_8bit = getattr(self.model, "is_loaded_in_8bit", False)
+        if loaded_in_8bit and not is_bnb_available():
+            raise ImportError(
+                "To use Adapter with 8-bit quantization, please install the `bitsandbytes` package. "
+                "You can install it with `pip install bitsandbytes`."
+            )
+        is_target_modules_in_base_model = False
+        is_hf_device_map_available = hasattr(self.model, "hf_device_map")
+        kwargs = {
+            "bottleneck_size": self.peft_config.bottleneck_size,
+            "non_linearity": self.peft_config.non_linearity,
+            "adapter_dropout": self.peft_config.adapter_dropout,
+            "scaling": self.peft_config.scaling,
+            "init_weights": self.peft_config.init_weights,
+        }
+        key_list = [key for key, _ in self.model.named_modules()]
+        for key in key_list:
+            if isinstance(self.peft_config.target_modules, str):
+                target_module_found = re.fullmatch(self.peft_config.target_modules, key)
+            else:
+                target_module_found = any(key.endswith(target_key) for target_key in self.peft_config.target_modules)
+            if target_module_found:
+                if not is_target_modules_in_base_model:
+                    is_target_modules_in_base_model = True
+                parent, target, target_name = self._get_submodules(key)
+                # determine the type of adapter to be used, this will effect the forward pass
+                if self.peft_config.use_parallel_adapter:
+                    adapter_type = "parallel_adapter"
+                else:
+                    adapter_type = TRANSFORMERS_MODELS_TO_ADAPTER_TYPE_MAPPING[self.model.config.model_type][target_name]
+                kwargs.update({"adapter_type": adapter_type})
+                    
+                bias = target.bias is not None
+                if loaded_in_8bit and isinstance(target, bnb.nn.Linear8bitLt):
+                    kwargs.update(
+                        {
+                            "has_fp16_weights": target.state.has_fp16_weights,
+                            "memory_efficient_backward": target.state.memory_efficient_backward,
+                            "threshold": target.state.threshold,
+                            "index": target.index,
+                        }
+                    )
+                    if adapter_type == "mh_adapter":
+                        new_module = Linear8bitLt(target.in_features, target.in_features, bias=bias, **kwargs)
+                    elif adapter_type == "output_adapter":
+                        new_module = Linear8bitLt(target.out_features, target.out_features, bias=bias, **kwargs)
+                    elif adapter_type == "parallel_adapter":
+                        new_module = Linear8bitLt(target.in_features, target.out_features, bias=bias, **kwargs)
+                elif isinstance(target, torch.nn.Linear):
+                    if adapter_type == "mh_adapter":
+                        new_module = Linear(target.in_features, target.in_features, bias=bias, **kwargs)
+                    elif adapter_type == "output_adapter":
+                        new_module = Linear(target.out_features, target.out_features, bias=bias, **kwargs)
+                    elif adapter_type == "parallel_adapter":
+                        new_module = Linear(target.in_features, target.out_features, bias=bias, **kwargs)
+                self._replace_module(parent, target_name, new_module, target)
+        if not is_target_modules_in_base_model:
+            raise ValueError(
+                f"Target modules {self.peft_config.target_modules} not found in the base model. "
+                f"Please check the target modules and try again."
+            )
+
+    def _get_submodules(self, key):
+        parent = self.model.get_submodule(".".join(key.split(".")[:-1]))
+        target_name = key.split(".")[-1]
+        target = self.model.get_submodule(key)
+        return parent, target, target_name
+
+    def _replace_module(self, parent_module, child_name, new_module, old_module):
+        setattr(parent_module, child_name, new_module)
+        new_module.weight = old_module.weight
+        if old_module.bias is not None:
+            new_module.bias = old_module.bias
+        if getattr(old_module, "state", None) is not None:
+            new_module.state = old_module.state
+            new_module.to(old_module.weight.device)
+    
+        # dispatch to correct device
+        for name, module in new_module.named_modules():
+            if "adapter_" in name:
+                module.to(old_module.weight.device)
+        
+    def __getattr__(self, name: str):
+        """Forward missing attributes to the wrapped module."""
+        try:
+            return super().__getattr__(name)  # defer to nn.Module's logic
+        except AttributeError:
+            return getattr(self.model, name)
+
+    @property
+    def modules_to_save(self):
+        return None
+
+    def get_peft_config_as_dict(self, inference: bool = False):
+        config = {k: v.value if isinstance(v, Enum) else v for k, v in asdict(self.peft_config).items()}
+        if inference:
+            config["inference_mode"] = True
+        return config
+
+    def _set_adapter_layers(self, enabled=True):
+        for module in self.model.modules():
+            if isinstance(module, AdapterLayer):
+                module.disable_adapters = False if enabled else True
+
+    def enable_adapter_layers(self):
+        self._set_adapter_layers(enabled=True)
+
+    def disable_adapter_layers(self):
+        self._set_adapter_layers(enabled=False)
+
+
+# Below code is based on https://github.com/adapter-hub/adapter-transformers/blob/master/src/transformers/adapters/modeling.py and lora.py from huggingfance PEFT
+# and modified to work with PyTorch FSDP
+
+
+#  ------------------------------------------------------------------------------------------
+#  Copyright (c) Microsoft Corporation. All rights reserved.
+#  Licensed under the MIT License (MIT). See LICENSE in the repo root for license information.
+#  ------------------------------------------------------------------------------------------
+
+
+# Copy from lora.py
+# had to adapt it for `lora_only` to work 
+def mark_only_adapter_as_trainable(model: nn.Module, bias: str = "none") -> None:
+    for n, p in model.named_parameters():
+        if "adapter_" not in n:
+            p.requires_grad = False
+    if bias == "none":
+        return
+    elif bias == "all":
+        for n, p in model.named_parameters():
+            if "bias" in n:
+                p.requires_grad = True
+    elif bias == "adapter_only":
+        for m in model.modules():
+            if isinstance(m, AdapterLayer) and hasattr(m, "bias") and m.bias is not None:
+                m.bias.requires_grad = True
+    else:
+        raise NotImplementedError
+
+
+class AdapterLayer:
+    def __init__(
+        self,
+        bottleneck_size: int,
+        non_linearity: str,
+        adapter_dropout: float,
+        scaling: Union[float, str],
+    ):
+        self.bottleneck_size = bottleneck_size
+        self.non_linearity = non_linearity
+        self.scaling = scaling
+        #optional dropout
+        if adapter_dropout > 0.0:
+            self.adapter_dropout = nn.Dropout(p=adapter_dropout)
+        else:
+            self.adapter_dropout = lambda x: x
+        self.disable_adapters = False
+
+
+class Linear(nn.Linear, AdapterLayer):
+    """
+    Bottleneck adapter in a dense layer. The adapter can be applied after the multi-head attention layer and/or
+    after the feed-forward layer.
+    """
+    def __init__(
+        self,
+        in_features: int,
+        out_features: int,
+        adapter_type: str,
+        bottleneck_size: int,
+        non_linearity: str,
+        adapter_dropout: float,
+        scaling: Union[float, str],
+        init_weights: str,
+        **kwargs,
+    ):
+        nn.Linear.__init__(self, in_features, out_features, **kwargs)
+        AdapterLayer.__init__(self, bottleneck_size=bottleneck_size,
+                                non_linearity=non_linearity,
+                                adapter_dropout=adapter_dropout,
+                                scaling=scaling)
+
+        self.init_weights = init_weights
+        self.adapter_type = adapter_type
+        if isinstance(scaling, float):
+            self.adapter_scaling = scaling
+        elif scaling == "learned":
+            self.adapter_scaling = nn.Parameter(torch.ones(1))
+        # Actual trainable parameters
+        self.adapter_down = nn.Linear(in_features, bottleneck_size, bias=False)
+        self.adapter_up = nn.Linear(bottleneck_size, out_features, bias=False)
+        self.act_fn = ACT2FN[self.non_linearity]
+        #Freezing the pre-trained weight matrix
+        self.weight.requires_grad = False
+        self.reset_parameters()
+
+    def reset_parameters(self):
+        nn.Linear.reset_parameters(self)
+        # if we want to initialize with the bert strategy then this function is called for all the linear layers
+        if hasattr(self, "adapter_down"):
+            if self.init_weights == "bert":
+                self.adapter_down.apply(self.init_bert_weights)
+                self.adapter_up.apply(self.init_bert_weights)
+            elif self.init_weights == "mam_adapter":
+                nn.init.kaiming_uniform_(self.adapter_down.weight, a=math.sqrt(5))
+                nn.init.zeros_(self.adapter_up.weight)
+            else:
+                raise ValueError("Unknown init_weights type: {}".format(config["init_weights"]))
+
+    # This is copied from the BertPreTrainedModel class to make this a self containing class.
+    @staticmethod
+    def init_bert_weights(module):
+        """Initialize the weights."""
+        if isinstance(module, (nn.Linear, nn.Embedding)):
+            # std defaults to 0.02, this might need to be changed
+            module.weight.data.normal_(mean=0.0, std=0.02)
+        elif isinstance(module, nn.LayerNorm):
+            module.bias.data.zero_()
+            module.weight.data.fill_(1.0)
+        if isinstance(module, nn.Linear) and module.bias is not None:
+            module.bias.data.zero_()
+
+    def train(self, mode: bool = True):
+        nn.Linear.train(self, mode)
+        self.adapter_down.train(mode)
+        self.adapter_up.train(mode)
+
+    def eval(self):
+        nn.Linear.eval(self)
+        self.adapter_down.eval()
+        self.adapter_up.eval()
+
+    def forward(self, x: torch.Tensor):
+        if self.disable_adapters:
+            return F.linear(x, self.weight, bias=self.bias)
+        else:
+            if self.adapter_type == "mh_adapter":
+                # for mh_adapter, x will pass the adapter first and then the linear layer
+                expected_dtype = x.dtype
+                residual = x
+
+                if x.dtype != torch.float32:
+                    x = x.float()
+                output = self.adapter_up(self.act_fn(self.adapter_down(self.adapter_dropout(x)))).to(expected_dtype) * self.adapter_scaling
+
+                output = output + residual
+                
+                result = F.linear(output, self.weight, bias=self.bias)
+            elif self.adapter_type == "output_adapter":
+                # for output_adapter, x will pass the linear layer first and then the adapter
+                x = F.linear(x, self.weight, bias=self.bias)
+                expected_dtype = x.dtype
+                residual = x
+
+                if x.dtype != torch.float32:
+                    x = x.float()
+
+                output = self.adapter_up(self.act_fn(self.adapter_down(self.adapter_dropout(x)))).to(expected_dtype) * self.adapter_scaling
+
+                result = output + residual
+            elif self.adapter_type == "parallel_adapter":
+                # for parallel_adapter, x will pass the linear layer first and the adapter layer parallelly. 
+                # The output of the adapter layer will be added to the output of the linear layer
+                result = F.linear(x, self.weight, bias=self.bias)
+                expected_dtype = result.dtype
+
+                if x.dtype != torch.float32:
+                    x = x.float()
+                output = self.adapter_up(self.act_fn(self.adapter_down(self.adapter_dropout(x)))).to(expected_dtype) * self.adapter_scaling
+
+                result = result + output
+            return result
+
+
+if is_bnb_available():
+
+    class Linear8bitLt(bnb.nn.Linear8bitLt, AdapterLayer):
+        # Aadapter layer for 8bit linear layer
+        def __init__(
+            self,
+            in_features: int,
+            out_features: int,
+            adapter_type: str,
+            bottleneck_size: int,
+            non_linearity: str,
+            adapter_dropout: float,
+            scaling: Union[float, str],
+            init_weights: str,
+            **kwargs,
+        ):
+            bnb.nn.Linear8bitLt.__init__(
+                self,
+                in_features,
+                out_features,
+                bias=kwargs.get("bias", True),
+                has_fp16_weights=kwargs.get("has_fp16_weights", True),
+                memory_efficient_backward=kwargs.get("memory_efficient_backward", False),
+                threshold=kwargs.get("threshold", 0.0),
+                index=kwargs.get("index", None),
+            )
+            AdapterLayer.__init__(
+                self, 
+                bottleneck_size=bottleneck_size, 
+                non_linearity=non_linearity, 
+                adapter_dropout=adapter_dropout,
+                scaling=scaling,)
+            
+            self.init_weights = init_weights
+            self.adapter_type = adapter_type
+            if isinstance(scaling, float):
+                self.adapter_scaling = scaling
+            elif scaling == "learned":
+                self.adapter_scaling = nn.Parameter(torch.ones(1))
+            # Actual trainable parameters
+            self.adapter_down = nn.Linear(in_features, bottleneck_size, bias=False)
+            self.adapter_up = nn.Linear(bottleneck_size, out_features, bias=False)
+            self.act_fn = ACT2FN[self.non_linearity]
+            #Freezing the pre-trained weight matrix
+            self.weight.requires_grad = False
+            self.reset_parameters()
+        
+        def reset_parameters(self):
+            nn.Linear.reset_parameters(self)
+            # if we want to initialize with the bert strategy then this function is called for all the linear layers
+            if hasattr(self, "adapter_down"):
+                if self.init_weights == "bert":
+                    self.adapter_down.apply(self.init_bert_weights)
+                    self.adapter_up.apply(self.init_bert_weights)
+                elif self.init_weights == "mam_adapter":
+                    nn.init.kaiming_uniform_(self.adapter_down.weight, a=math.sqrt(5))
+                    nn.init.zeros_(self.adapter_up.weight)
+                else:
+                    raise ValueError("Unknown init_weights type: {}".format(config["init_weights"]))
+
+        # This is copied from the BertPreTrainedModel class to make this a self containing class.
+        @staticmethod
+        def init_bert_weights(module):
+            """Initialize the weights."""
+            if isinstance(module, (nn.Linear, nn.Embedding)):
+                # std defaults to 0.02, this might need to be changed
+                module.weight.data.normal_(mean=0.0, std=0.02)
+            elif isinstance(module, nn.LayerNorm):
+                module.bias.data.zero_()
+                module.weight.data.fill_(1.0)
+            if isinstance(module, nn.Linear) and module.bias is not None:
+                module.bias.data.zero_()
+        
+        def forward(self, x: torch.Tensor):
+            result_pre_forward = super().forward(x)
+
+            if self.disable_adapters:
+                return result_pre_forward
+            else:
+                if self.adapter_type == "mh_adapter":
+                    if not torch.is_autocast_enabled():
+                        expected_dtype = x.dtype
+
+                        if x.dtype != torch.float32:
+                            x = x.float()
+                        
+                        residual = x
+                        output = self.adapter_up(self.act_fn(self.adapter_down(self.adapter_dropout(x)))).to(expected_dtype) * self.adapter_scaling
+                        output = (output + residual).to(expected_dtype)
+
+                        result = super().forward(output)
+                    else:
+                        residual = x
+                        output = self.adapter_up(self.act_fn(self.adapter_down(self.adapter_dropout(x)))) * self.adapter_scaling
+                        output = output + residual
+
+                        result = super().forward(output)
+                elif self.adapter_type == "output_adapter":
+                    if not torch.is_autocast_enabled():
+                        expected_dtype = result_pre_forward.dtype
+
+                        if result_pre_forward.dtype != torch.float32:
+                            result_pre_forward = result_pre_forward.float()
+
+                        residual = result_pre_forward
+                        output = self.adapter_up(self.act_fn(self.adapter_down(self.adapter_dropout(result_pre_forward)))).to(expected_dtype) * self.adapter_scaling
+                        result = (output + residual).to(expected_dtype)
+                    else:
+                        residual = result_pre_forward
+                        output = self.adapter_up(self.act_fn(self.adapter_down(self.adapter_dropout(result_pre_forward)))) * self.adapter_scaling
+                        result = output + residual
+                elif self.adapter_type == "parallel_adapter":
+                    if not torch.is_autocast_enabled():
+                        expected_dtype = result_pre_forward.dtype
+
+                        if x.dtype != torch.float32:
+                            x = x.float()
+                        
+                        output = self.adapter_up(self.act_fn(self.adapter_down(self.adapter_dropout(x)))).to(expected_dtype) * self.adapter_scaling
+                        result = result_pre_forward + output
+                    else:
+                        output = self.adapter_up(self.act_fn(self.adapter_down(self.adapter_dropout(x)))) * self.adapter_scaling
+                        result = result_pre_forward + output
+
+                return result
+                        
+
+                        
+
+
+
+            
+
+
+
+
+
+
+
+        

SVFT-main/LLM-Adapters/peft/src/peft/tuners/lora.py

@@ -0,0 +1,624 @@
+# coding=utf-8
+# Copyright 2023-present the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import importlib
+import math
+import re
+import warnings
+from dataclasses import asdict, dataclass, field
+from enum import Enum
+from typing import List, Optional, Union
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from transformers.pytorch_utils import Conv1D
+
+from ..utils import PeftConfig, PeftType, transpose
+
+
+def is_bnb_available():
+    return importlib.util.find_spec("bitsandbytes") is not None
+
+
+if is_bnb_available():
+    import bitsandbytes as bnb
+
+
+@dataclass
+class LoraConfig(PeftConfig):
+    """
+    This is the configuration class to store the configuration of a [`~peft.Lora`].
+
+    Args:
+        r (`int`): Lora attention dimension
+        target_modules (`Union[List[str],str]`): The names of the modules to apply Lora to.
+        lora_alpha (`float`): The alpha parameter for Lora scaling.
+        lora_dropout (`float`): The dropout probability for Lora layers.
+        merge_weights (`bool`):
+            Whether to merge the weights of the Lora layers with the base transformer model in `eval` mode.
+        fan_in_fan_out (`bool`): Set this to True if the layer to replace stores weight like (fan_in, fan_out)
+        enable_lora ( `List[bool]`): Used with `lora.MergedLinear`.
+        bias (`str`): Bias type for Lora. Can be 'none', 'all' or 'lora_only'
+        modules_to_save (`List[str]`):List of modules apart from LoRA layers to be set as trainable
+            and saved in the final checkpoint.
+    """
+
+    r: int = field(default=8, metadata={"help": "Lora attention dimension"})
+    target_modules: Optional[Union[List[str], str]] = field(
+        default=None,
+        metadata={
+            "help": "List of module names or regex expression of the module names to replace with Lora."
+            "For example, ['q', 'v'] or '.*decoder.*(SelfAttention|EncDecAttention).*(q|v)$' "
+        },
+    )
+    lora_alpha: int = field(default=None, metadata={"help": "Lora alpha"})
+    lora_dropout: float = field(default=None, metadata={"help": "Lora dropout"})
+    merge_weights: bool = field(
+        default=False, metadata={"help": "Merge weights of the original model and the Lora model"}
+    )
+    fan_in_fan_out: bool = field(
+        default=False,
+        metadata={"help": "Set this to True if the layer to replace stores weight like (fan_in, fan_out)"},
+    )
+    enable_lora: Optional[List[bool]] = field(default=None, metadata={"help": "Used with `lora.MergedLinear`."})
+    bias: str = field(default="none", metadata={"help": "Bias type for Lora. Can be 'none', 'all' or 'lora_only'"})
+    modules_to_save: Optional[List[str]] = field(
+        default=None,
+        metadata={
+            "help": "List of modules apart from LoRA layers to be set as trainable and saved in the final checkpoint. "
+            "For example, in Sequence Classification or Token Classification tasks, "
+            "the final layer `classifier/score` are randomly initialized and as such need to be trainable and saved."
+        },
+    )
+
+    def __post_init__(self):
+        self.peft_type = PeftType.LORA
+
+
+class LoraModel(torch.nn.Module):
+    """
+    Creates Low Rank Adapter (Lora) model from a pretrained transformers model.
+
+    Args:
+        model ([`transformers.PreTrainedModel`]): The model to be adapted.
+        config ([`LoraConfig`]): The configuration of the Lora model.
+
+    Returns:
+        `torch.nn.Module`: The Lora model.
+
+    Example::
+
+        >>> from transformers import AutoModelForSeq2SeqLM, LoraConfig >>> from peft import LoraModel, LoraConfig >>>
+        config = LoraConfig(
+            peft_type="LORA", task_type="SEQ_2_SEQ_LM", r=8, lora_alpha=32, target_modules=["q", "v"],
+            lora_dropout=0.01, )
+        >>> model = AutoModelForSeq2SeqLM.from_pretrained("t5-base") >>> lora_model = LoraModel(config, model)
+
+    **Attributes**:
+        - **model** ([`transformers.PreTrainedModel`]) -- The model to be adapted.
+        - **peft_config** ([`LoraConfig`]): The configuration of the Lora model.
+    """
+
+    def __init__(self, config, model):
+        super().__init__()
+        self.peft_config = config
+        self.model = model
+        self._find_and_replace()
+        mark_only_lora_as_trainable(self.model, self.peft_config.bias)
+        self.forward = self.model.forward
+
+    def _find_and_replace(self):
+        loaded_in_8bit = getattr(self.model, "is_loaded_in_8bit", False)
+        if loaded_in_8bit and not is_bnb_available():
+            raise ImportError(
+                "To use Lora with 8-bit quantization, please install the `bitsandbytes` package. "
+                "You can install it with `pip install bitsandbytes`."
+            )
+        is_target_modules_in_base_model = False
+        is_hf_device_map_available = hasattr(self.model, "hf_device_map")
+        kwargs = {
+            "r": self.peft_config.r,
+            "lora_alpha": self.peft_config.lora_alpha,
+            "lora_dropout": self.peft_config.lora_dropout,
+            "fan_in_fan_out": self.peft_config.fan_in_fan_out,
+            "merge_weights": (self.peft_config.merge_weights or self.peft_config.inference_mode)
+            and not is_hf_device_map_available,
+        }
+        key_list = [key for key, _ in self.model.named_modules()]
+        for key in key_list:
+            if isinstance(self.peft_config.target_modules, str):
+                target_module_found = re.fullmatch(self.peft_config.target_modules, key)
+            else:
+                target_module_found = any(key.endswith(target_key) for target_key in self.peft_config.target_modules)
+            if target_module_found:
+                if not is_target_modules_in_base_model:
+                    is_target_modules_in_base_model = True
+                parent, target, target_name = self._get_submodules(key)
+                bias = target.bias is not None
+                if loaded_in_8bit and isinstance(target, bnb.nn.Linear8bitLt):
+                    kwargs.update(
+                        {
+                            "has_fp16_weights": target.state.has_fp16_weights,
+                            "memory_efficient_backward": target.state.memory_efficient_backward,
+                            "threshold": target.state.threshold,
+                            "index": target.index,
+                        }
+                    )
+                    if self.peft_config.enable_lora is None:
+                        new_module = Linear8bitLt(target.in_features, target.out_features, bias=bias, **kwargs)
+                    else:
+                        kwargs.update({"enable_lora": self.peft_config.enable_lora})
+                        new_module = MergedLinear8bitLt(target.in_features, target.out_features, bias=bias, **kwargs)
+                elif isinstance(target, torch.nn.Linear) and self.peft_config.enable_lora is None:
+                    new_module = Linear(target.in_features, target.out_features, bias=bias, **kwargs)
+                elif self.peft_config.enable_lora is not None:
+                    kwargs.update({"enable_lora": self.peft_config.enable_lora})
+                    if isinstance(target, Conv1D):
+                        in_features, out_features = (
+                            target.weight.ds_shape if hasattr(target.weight, "ds_shape") else target.weight.shape
+                        )
+                    else:
+                        in_features, out_features = target.in_features, target.out_features
+                        if kwargs["fan_in_fan_out"]:
+                            warnings.warn(
+                                "fan_in_fan_out is set to True but the target module is not a Conv1D. "
+                                "Setting fan_in_fan_out to False."
+                            )
+                            kwargs["fan_in_fan_out"] = self.peft_config.fan_in_fan_out = False
+                    new_module = MergedLinear(in_features, out_features, bias=bias, **kwargs)
+                self._replace_module(parent, target_name, new_module, target)
+        if not is_target_modules_in_base_model:
+            raise ValueError(
+                f"Target modules {self.peft_config.target_modules} not found in the base model. "
+                f"Please check the target modules and try again."
+            )
+
+    def _get_submodules(self, key):
+        parent = self.model.get_submodule(".".join(key.split(".")[:-1]))
+        target_name = key.split(".")[-1]
+        target = self.model.get_submodule(key)
+        return parent, target, target_name
+
+    def _replace_module(self, parent_module, child_name, new_module, old_module):
+        setattr(parent_module, child_name, new_module)
+        new_module.weight = old_module.weight
+        if old_module.bias is not None:
+            new_module.bias = old_module.bias
+        if getattr(old_module, "state", None) is not None:
+            new_module.state = old_module.state
+            new_module.to(old_module.weight.device)
+
+        # dispatch to correct device
+        for name, module in new_module.named_modules():
+            if "lora_" in name:
+                module.to(old_module.weight.device)
+
+    def __getattr__(self, name: str):
+        """Forward missing attributes to the wrapped module."""
+        try:
+            return super().__getattr__(name)  # defer to nn.Module's logic
+        except AttributeError:
+            return getattr(self.model, name)
+
+    @property
+    def modules_to_save(self):
+        return None
+
+    def get_peft_config_as_dict(self, inference: bool = False):
+        config = {k: v.value if isinstance(v, Enum) else v for k, v in asdict(self.peft_config).items()}
+        if inference:
+            config["inference_mode"] = True
+        return config
+
+    def _set_adapter_layers(self, enabled=True):
+        for module in self.model.modules():
+            if isinstance(module, LoraLayer):
+                module.disable_adapters = False if enabled else True
+
+    def enable_adapter_layers(self):
+        self._set_adapter_layers(enabled=True)
+
+    def disable_adapter_layers(self):
+        self._set_adapter_layers(enabled=False)
+
+
+# Below code is based on https://github.com/microsoft/LoRA/blob/main/loralib/layers.py
+# and modified to work with PyTorch FSDP
+
+
+#  ------------------------------------------------------------------------------------------
+#  Copyright (c) Microsoft Corporation. All rights reserved.
+#  Licensed under the MIT License (MIT). See LICENSE in the repo root for license information.
+#  ------------------------------------------------------------------------------------------
+
+
+# had to adapt it for `lora_only` to work
+def mark_only_lora_as_trainable(model: nn.Module, bias: str = "none") -> None:
+    for n, p in model.named_parameters():
+        if "lora_" not in n:
+            p.requires_grad = False
+    if bias == "none":
+        return
+    elif bias == "all":
+        for n, p in model.named_parameters():
+            if "bias" in n:
+                p.requires_grad = True
+    elif bias == "lora_only":
+        for m in model.modules():
+            if isinstance(m, LoraLayer) and hasattr(m, "bias") and m.bias is not None:
+                m.bias.requires_grad = True
+    else:
+        raise NotImplementedError
+
+
+class LoraLayer:
+    def __init__(
+        self,
+        r: int,
+        lora_alpha: int,
+        lora_dropout: float,
+        merge_weights: bool,
+    ):
+        self.r = r
+        self.lora_alpha = lora_alpha
+        # Optional dropout
+        if lora_dropout > 0.0:
+            self.lora_dropout = nn.Dropout(p=lora_dropout)
+        else:
+            self.lora_dropout = lambda x: x
+        # Mark the weight as unmerged
+        self.merged = False
+        self.merge_weights = merge_weights
+        self.disable_adapters = False
+
+
+class Linear(nn.Linear, LoraLayer):
+    # Lora implemented in a dense layer
+    def __init__(
+        self,
+        in_features: int,
+        out_features: int,
+        r: int = 0,
+        lora_alpha: int = 1,
+        lora_dropout: float = 0.0,
+        fan_in_fan_out: bool = False,  # Set this to True if the layer to replace stores weight like (fan_in, fan_out)
+        merge_weights: bool = True,
+        **kwargs,
+    ):
+        nn.Linear.__init__(self, in_features, out_features, **kwargs)
+        LoraLayer.__init__(self, r=r, lora_alpha=lora_alpha, lora_dropout=lora_dropout, merge_weights=merge_weights)
+
+        self.fan_in_fan_out = fan_in_fan_out
+        # Actual trainable parameters
+        if r > 0:
+            self.lora_A = nn.Linear(in_features, r, bias=False)
+            self.lora_B = nn.Linear(r, out_features, bias=False)
+            self.scaling = self.lora_alpha / self.r
+            # Freezing the pre-trained weight matrix
+            self.weight.requires_grad = False
+        self.reset_parameters()
+        if fan_in_fan_out:
+            self.weight.data = self.weight.data.T
+
+    def reset_parameters(self):
+        nn.Linear.reset_parameters(self)
+        if hasattr(self, "lora_A"):
+            # initialize A the same way as the default for nn.Linear and B to zero
+            nn.init.kaiming_uniform_(self.lora_A.weight, a=math.sqrt(5))
+            nn.init.zeros_(self.lora_B.weight)
+
+    def train(self, mode: bool = True):
+        nn.Linear.train(self, mode)
+        self.lora_A.train(mode)
+        self.lora_B.train(mode)
+        if not mode and self.merge_weights and not self.merged:
+            # Merge the weights and mark it
+            if self.r > 0:
+                self.weight.data += (
+                    transpose(self.lora_B.weight @ self.lora_A.weight, self.fan_in_fan_out) * self.scaling
+                )
+            self.merged = True
+        elif self.merge_weights and self.merged:
+            # Make sure that the weights are not merged
+            if self.r > 0:
+                self.weight.data -= (
+                    transpose(self.lora_B.weight @ self.lora_A.weight, self.fan_in_fan_out) * self.scaling
+                )
+            self.merged = False
+
+    def eval(self):
+        nn.Linear.eval(self)
+        self.lora_A.eval()
+        self.lora_B.eval()
+
+    def forward(self, x: torch.Tensor):
+        previous_dtype = self.weight.dtype
+
+        if self.disable_adapters:
+            if self.r > 0 and self.merged:
+                matmul_output = self.lora_B.weight @ self.lora_A.weight
+                self.weight.data -= transpose(matmul_output.to(previous_dtype), self.fan_in_fan_out) * self.scaling
+                self.merged = False
+
+            result = F.linear(x, transpose(self.weight, self.fan_in_fan_out), bias=self.bias)
+        elif self.r > 0 and not self.merged:
+            result = F.linear(x, transpose(self.weight, self.fan_in_fan_out), bias=self.bias)
+            if self.r > 0:
+                result += self.lora_B(self.lora_A(self.lora_dropout(x.to(self.lora_A.weight.dtype)))) * self.scaling
+        else:
+             result = F.linear(x, transpose(self.weight, self.fan_in_fan_out), bias=self.bias)
+
+        if result.dtype != previous_dtype:
+            result = result.to(previous_dtype)
+
+        return result
+
+class MergedLinear(nn.Linear, LoraLayer):
+    # Lora implemented in a dense layer
+    def __init__(
+        self,
+        in_features: int,
+        out_features: int,
+        r: int = 0,
+        lora_alpha: int = 1,
+        lora_dropout: float = 0.0,
+        enable_lora: List[bool] = [False],
+        fan_in_fan_out: bool = False,
+        merge_weights: bool = True,
+        **kwargs,
+    ):
+        nn.Linear.__init__(self, in_features, out_features, **kwargs)
+        LoraLayer.__init__(self, r=r, lora_alpha=lora_alpha, lora_dropout=lora_dropout, merge_weights=merge_weights)
+        if out_features % len(enable_lora) != 0:
+            raise ValueError("The length of enable_lora must divide out_features")
+        self.enable_lora = enable_lora
+        self.fan_in_fan_out = fan_in_fan_out
+        # Actual trainable parameters
+        if r > 0 and any(enable_lora):
+            self.lora_A = nn.Linear(in_features, r * sum(enable_lora), bias=False)
+            self.lora_B = nn.Conv1d(
+                r * sum(enable_lora),
+                out_features // len(enable_lora) * sum(enable_lora),
+                kernel_size=1,
+                groups=2,
+                bias=False,
+            )
+            self.scaling = self.lora_alpha / self.r
+            # Freezing the pre-trained weight matrix
+            self.weight.requires_grad = False
+            # Compute the indices
+            self.lora_ind = self.weight.new_zeros((out_features,), dtype=torch.bool).view(len(enable_lora), -1)
+            self.lora_ind[enable_lora, :] = True
+            self.lora_ind = self.lora_ind.view(-1)
+        self.reset_parameters()
+        if fan_in_fan_out:
+            self.weight.data = self.weight.data.T
+
+    def reset_parameters(self):
+        nn.Linear.reset_parameters(self)
+        if hasattr(self, "lora_A"):
+            # initialize A the same way as the default for nn.Linear and B to zero
+            nn.init.kaiming_uniform_(self.lora_A.weight, a=math.sqrt(5))
+            nn.init.zeros_(self.lora_B.weight)
+
+    def zero_pad(self, x):
+        result = x.new_zeros((*x.shape[:-1], self.out_features))
+        result = result.view(-1, self.out_features)
+        result[:, self.lora_ind] = x.reshape(-1, self.out_features // len(self.enable_lora) * sum(self.enable_lora))
+        return result.view((*x.shape[:-1], self.out_features))
+
+    def train(self, mode: bool = True):
+        nn.Linear.train(self, mode)
+        self.lora_A.train(mode)
+        self.lora_B.train(mode)
+        if not mode and self.merge_weights and not self.merged:
+            # Merge the weights and mark it
+            if self.r > 0 and any(self.enable_lora):
+                delta_w = (
+                    F.conv1d(
+                        self.lora_A.weight.data.unsqueeze(0),
+                        self.lora_B.weight.data,
+                        groups=sum(self.enable_lora),
+                    )
+                    .squeeze(0)
+                    .transpose(-2, -1)
+                )
+                self.weight.data += transpose(self.zero_pad(delta_w * self.scaling), not self.fan_in_fan_out)
+            self.merged = True
+        elif self.merge_weights and self.merged:
+            # Make sure that the weights are not merged
+            if self.r > 0 and any(self.enable_lora):
+                delta_w = (
+                    F.conv1d(
+                        self.lora_A.weight.data.unsqueeze(0),
+                        self.lora_B.weight.data,
+                        groups=sum(self.enable_lora),
+                    )
+                    .squeeze(0)
+                    .transpose(-2, -1)
+                )
+                self.weight.data -= transpose(self.zero_pad(delta_w * self.scaling), not self.fan_in_fan_out)
+            self.merged = False
+
+    def eval(self):
+        nn.Linear.eval(self)
+        self.lora_A.eval()
+        self.lora_B.eval()
+
+    def forward(self, x: torch.Tensor):
+        previous_dtype = x.dtype
+        if self.disable_adapters:
+            if self.r > 0 and self.merged and any(self.enable_lora):
+                delta_w = (
+                    F.conv1d(
+                        self.lora_A.weight.data.unsqueeze(0),
+                        self.lora_B.weight.data,
+                        groups=sum(self.enable_lora),
+                    )
+                    .squeeze(0)
+                    .transpose(-2, -1)
+                )
+                delta_w = delta_w.to(self.weight.dtype)
+                self.weight.data -= transpose(self.zero_pad(delta_w * self.scaling), not self.fan_in_fan_out)
+                self.merged = False
+            result = F.linear(x, transpose(self.weight, self.fan_in_fan_out), bias=self.bias)
+        elif self.merged:
+            result = F.linear(x, transpose(self.weight, self.fan_in_fan_out), bias=self.bias)
+        else:
+            result = F.linear(x, transpose(self.weight, self.fan_in_fan_out), bias=self.bias)
+            if self.r > 0:
+                after_A = self.lora_A(self.lora_dropout(x.to(self.lora_A.weight.dtype)))
+                after_B = self.lora_B(after_A.transpose(-2, -1)).transpose(-2, -1)
+                result += self.zero_pad(after_B) * self.scaling
+        result = result.to(previous_dtype)
+
+        return result
+
+
+if is_bnb_available():
+
+    class Linear8bitLt(bnb.nn.Linear8bitLt, LoraLayer):
+        # Lora implemented in a dense layer
+        def __init__(
+            self,
+            in_features,
+            out_features,
+            r: int = 0,
+            lora_alpha: int = 1,
+            lora_dropout: float = 0.0,
+            **kwargs,
+        ):
+            bnb.nn.Linear8bitLt.__init__(
+                self,
+                in_features,
+                out_features,
+                bias=kwargs.get("bias", True),
+                has_fp16_weights=kwargs.get("has_fp16_weights", True),
+                memory_efficient_backward=kwargs.get("memory_efficient_backward", False),
+                threshold=kwargs.get("threshold", 0.0),
+                index=kwargs.get("index", None),
+            )
+            LoraLayer.__init__(self, r=r, lora_alpha=lora_alpha, lora_dropout=lora_dropout, merge_weights=False)
+            # Actual trainable parameters
+            if r > 0:
+                self.lora_A = nn.Linear(in_features, r, bias=False)
+                self.lora_B = nn.Linear(r, out_features, bias=False)
+                self.scaling = self.lora_alpha / self.r
+                # Freezing the pre-trained weight matrix
+                self.weight.requires_grad = False
+            self.reset_parameters()
+
+        def reset_parameters(self):
+            if hasattr(self, "lora_A"):
+                # initialize A the same way as the default for nn.Linear and B to zero
+                nn.init.kaiming_uniform_(self.lora_A.weight, a=math.sqrt(5))
+                nn.init.zeros_(self.lora_B.weight)
+
+        def forward(self, x: torch.Tensor):
+            result = super().forward(x)
+
+            if self.disable_adapters:
+                return result
+            elif self.r > 0:
+                if not torch.is_autocast_enabled():
+                    expected_dtype = result.dtype
+
+                    if x.dtype != torch.float32:
+                        x = x.float()
+                    output = self.lora_B(self.lora_A(self.lora_dropout(x))).to(expected_dtype) * self.scaling
+                    result += output
+                else:
+                    output = self.lora_B(self.lora_A(self.lora_dropout(x))) * self.scaling
+                    result += output
+            return result
+
+    class MergedLinear8bitLt(bnb.nn.Linear8bitLt, LoraLayer):
+        # Lora implemented in a dense layer
+        def __init__(
+            self,
+            in_features: int,
+            out_features: int,
+            r: int = 0,
+            lora_alpha: int = 1,
+            lora_dropout: float = 0.0,
+            enable_lora: List[bool] = [False],
+            **kwargs,
+        ):
+            bnb.nn.Linear8bitLt.__init__(
+                self,
+                in_features,
+                out_features,
+                bias=kwargs.get("bias", True),
+                has_fp16_weights=kwargs.get("has_fp16_weights", True),
+                memory_efficient_backward=kwargs.get("memory_efficient_backward", False),
+                threshold=kwargs.get("threshold", 0.0),
+                index=kwargs.get("index", None),
+            )
+            LoraLayer.__init__(self, r=r, lora_alpha=lora_alpha, lora_dropout=lora_dropout, merge_weights=False)
+            if out_features % len(enable_lora) != 0:
+                raise ValueError("The length of enable_lora must divide out_features")
+            self.enable_lora = enable_lora
+            # Actual trainable parameters
+            if r > 0 and any(enable_lora):
+                self.lora_A = nn.Linear(in_features, r * sum(enable_lora), bias=False)
+                self.lora_B = nn.Conv1d(
+                    r * sum(enable_lora),
+                    out_features // len(enable_lora) * sum(enable_lora),
+                    kernel_size=1,
+                    groups=2,
+                    bias=False,
+                )
+                self.scaling = self.lora_alpha / self.r
+                # Freezing the pre-trained weight matrix
+                self.weight.requires_grad = False
+                # Compute the indices
+                self.lora_ind = self.weight.new_zeros((out_features,), dtype=torch.bool).view(len(enable_lora), -1)
+                self.lora_ind[enable_lora, :] = True
+                self.lora_ind = self.lora_ind.view(-1)
+            self.reset_parameters()
+
+        def reset_parameters(self):
+            if hasattr(self, "lora_A"):
+                # initialize A the same way as the default for nn.Linear and B to zero
+                nn.init.kaiming_uniform_(self.lora_A.weight, a=math.sqrt(5))
+                nn.init.zeros_(self.lora_B.weight)
+
+        def zero_pad(self, x):
+            result = x.new_zeros((*x.shape[:-1], self.out_features))
+            result = result.view(-1, self.out_features)
+            result[:, self.lora_ind] = x.reshape(
+                -1, self.out_features // len(self.enable_lora) * sum(self.enable_lora)
+            )
+            return result.view((*x.shape[:-1], self.out_features))
+
+        def forward(self, x: torch.Tensor):
+            result = super().forward(x)
+            if self.disable_adapters:
+                return result
+            elif self.r > 0:
+                if not torch.is_autocast_enabled():
+                    expected_dtype = result.dtype
+                    if x.dtype != torch.float32:
+                        x = x.float()
+                    after_A = self.lora_A(self.lora_dropout(x))
+                    after_B = self.lora_B(after_A.transpose(-2, -1)).transpose(-2, -1)
+                    output = self.zero_pad(after_B).to(expected_dtype) * self.scaling
+                    result += output
+                else:
+                    after_A = self.lora_A(self.lora_dropout(x))
+                    after_B = self.lora_B(after_A.transpose(-2, -1)).transpose(-2, -1)
+                    output = self.zero_pad(after_B) * self.scaling
+                    result += output
+            return result

SVFT-main/LLM-Adapters/peft/src/peft/tuners/p_tuning.py

@@ -0,0 +1,159 @@
+# coding=utf-8
+# Copyright 2023-present the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import enum
+import warnings
+from dataclasses import dataclass, field
+from typing import Union
+
+import torch
+
+from ..utils import PeftType, PromptLearningConfig
+
+
+class PromptEncoderReparameterizationType(str, enum.Enum):
+    MLP = "MLP"
+    LSTM = "LSTM"
+
+
+@dataclass
+class PromptEncoderConfig(PromptLearningConfig):
+    """
+    This is the configuration class to store the configuration of a [`~peft.PromptEncoder`].
+
+    Args:
+        encoder_reparameterization_type
+            (Union[[`PromptEncoderReparameterizationType`], `str`]): The type of reparameterization to use.
+        encoder_hidden_size (`int`): The hidden size of the prompt encoder.
+        encoder_num_layers (`int`): The number of layers of the prompt encoder.
+        encoder_dropout (`float`): The dropout probability of the prompt encoder.
+    """
+
+    encoder_reparameterization_type: Union[str, PromptEncoderReparameterizationType] = field(
+        default=PromptEncoderReparameterizationType.MLP,
+        metadata={"help": "How to reparameterize the prompt encoder"},
+    )
+    encoder_hidden_size: int = field(
+        default=None,
+        metadata={"help": "The hidden size of the prompt encoder"},
+    )
+    encoder_num_layers: int = field(
+        default=2,
+        metadata={"help": "The number of layers of the prompt encoder"},
+    )
+    encoder_dropout: float = field(
+        default=0.0,
+        metadata={"help": "The dropout of the prompt encoder"},
+    )
+
+    def __post_init__(self):
+        self.peft_type = PeftType.P_TUNING
+
+
+# Based on https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/nlp/modules/common/prompt_encoder.py
+# with some refactor
+class PromptEncoder(torch.nn.Module):
+    """
+    The prompt encoder network that is used to generate the virtual token embeddings for p-tuning.
+
+    Args:
+        config ([`PromptEncoderConfig`]): The configuration of the prompt encoder.
+
+    Example::
+
+        >>> from peft import PromptEncoder, PromptEncoderConfig >>> config = PromptEncoderConfig(
+                peft_type="P_TUNING", task_type="SEQ_2_SEQ_LM", num_virtual_tokens=20, token_dim=768,
+                num_transformer_submodules=1, num_attention_heads=12, num_layers=12,
+                encoder_reparameterization_type="MLP", encoder_hidden_size=768
+            )
+        >>> prompt_encoder = PromptEncoder(config)
+
+    **Attributes**:
+        - **embedding** ([`~torch.nn.Embedding`]) -- The embedding layer of the prompt encoder.
+        - **mlp_head** ([`~torch.nn.Sequential`]) -- The MLP head of the prompt encoder if `inference_mode=False`.
+        - **lstm_head** ([`~torch.nn.LSTM`]) -- The LSTM head of the prompt encoder if `inference_mode=False` and
+        `encoder_reparameterization_type="LSTM"`.
+        - **token_dim** (`int`) -- The hidden embedding dimension of the base transformer model.
+        - **input_size** (`int`) -- The input size of the prompt encoder.
+        - **output_size** (`int`) -- The output size of the prompt encoder.
+        - **hidden_size** (`int`) -- The hidden size of the prompt encoder.
+        - **total_virtual_tokens** (`int`): The total number of virtual tokens of the
+        prompt encoder.
+        - **encoder_type** (Union[[`PromptEncoderReparameterizationType`], `str`]):
+            The encoder type of the prompt encoder.
+
+
+    Input shape: (batch_size, total_virtual_tokens)
+
+    Output shape: (batch_size, total_virtual_tokens, token_dim)
+    """
+
+    def __init__(self, config):
+        super().__init__()
+        self.token_dim = config.token_dim
+        self.input_size = self.token_dim
+        self.output_size = self.token_dim
+        self.hidden_size = config.encoder_hidden_size
+        self.total_virtual_tokens = config.num_virtual_tokens * config.num_transformer_submodules
+        self.encoder_type = config.encoder_reparameterization_type
+
+        # embedding
+        self.embedding = torch.nn.Embedding(self.total_virtual_tokens, self.token_dim)
+        if not config.inference_mode:
+            if self.encoder_type == PromptEncoderReparameterizationType.LSTM:
+                lstm_dropout = config.encoder_dropout
+                num_layers = config.encoder_num_layers
+                # LSTM
+                self.lstm_head = torch.nn.LSTM(
+                    input_size=self.input_size,
+                    hidden_size=self.hidden_size,
+                    num_layers=num_layers,
+                    dropout=lstm_dropout,
+                    bidirectional=True,
+                    batch_first=True,
+                )
+
+                self.mlp_head = torch.nn.Sequential(
+                    torch.nn.Linear(self.hidden_size * 2, self.hidden_size * 2),
+                    torch.nn.ReLU(),
+                    torch.nn.Linear(self.hidden_size * 2, self.output_size),
+                )
+
+            elif self.encoder_type == PromptEncoderReparameterizationType.MLP:
+                warnings.warn(
+                    f"for {self.encoder_type}, the `encoder_num_layers` is ignored. Exactly 2 MLP layers are used."
+                )
+                layers = [
+                    torch.nn.Linear(self.input_size, self.hidden_size),
+                    torch.nn.ReLU(),
+                    torch.nn.Linear(self.hidden_size, self.hidden_size),
+                    torch.nn.ReLU(),
+                    torch.nn.Linear(self.hidden_size, self.output_size),
+                ]
+                self.mlp_head = torch.nn.Sequential(*layers)
+
+            else:
+                raise ValueError("Prompt encoder type not recognized. Please use one of MLP (recommended) or LSTM.")
+
+    def forward(self, indices):
+        input_embeds = self.embedding(indices)
+        if self.encoder_type == PromptEncoderReparameterizationType.LSTM:
+            output_embeds = self.mlp_head(self.lstm_head(input_embeds)[0])
+        elif self.encoder_type == PromptEncoderReparameterizationType.MLP:
+            output_embeds = self.mlp_head(input_embeds)
+        else:
+            raise ValueError("Prompt encoder type not recognized. Please use one of MLP (recommended) or LSTM.")
+
+        return output_embeds

SVFT-main/LLM-Adapters/peft/src/peft/tuners/prefix_tuning.py

@@ -0,0 +1,101 @@
+# coding=utf-8
+# Copyright 2023-present the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+from dataclasses import dataclass, field
+
+import torch
+
+from ..utils import PeftType, PromptLearningConfig
+
+
+@dataclass
+class PrefixTuningConfig(PromptLearningConfig):
+    """
+    This is the configuration class to store the configuration of a [`~peft.PrefixEncoder`].
+
+    Args:
+        encoder_hidden_size (`int`): The hidden size of the prompt encoder.
+        prefix_projection (`bool`): Whether to project the prefix embeddings.
+    """
+
+    encoder_hidden_size: int = field(
+        default=None,
+        metadata={"help": "The hidden size of the encoder"},
+    )
+    prefix_projection: bool = field(
+        default=False,
+        metadata={"help": "Whether to project the prefix tokens"},
+    )
+
+    def __post_init__(self):
+        self.peft_type = PeftType.PREFIX_TUNING
+
+
+# Based on https://github.com/THUDM/P-tuning-v2/blob/main/model/prefix_encoder.py
+# with some refactor
+class PrefixEncoder(torch.nn.Module):
+    r"""
+    The torch.nn model to encode the prefix
+
+    Args:
+        config ([`PrefixTuningConfig`]): The configuration of the prefix encoder.
+
+    Example::
+
+        >>> from peft import PrefixEncoder, PrefixTuningConfig >>> config = PrefixTuningConfig(
+                peft_type="PREFIX_TUNING", task_type="SEQ_2_SEQ_LM", num_virtual_tokens=20, token_dim=768,
+                num_transformer_submodules=1, num_attention_heads=12, num_layers=12, encoder_hidden_size=768
+            )
+        >>> prefix_encoder = PrefixEncoder(config)
+
+
+    **Attributes**:
+        - **embedding** (`torch.nn.Embedding`) --
+            The embedding layer of the prefix encoder.
+        - **transform** (`torch.nn.Sequential`) -- The
+        two-layer MLP to transform the prefix embeddings if `prefix_projection` is `True`.
+        - **prefix_projection** (`bool`) -- Whether to project the prefix embeddings.
+
+    Input shape: (batch_size, num_virtual_tokens)
+
+    Output shape: (batch_size, num_virtual_tokens, 2*layers*hidden)
+    """
+
+    def __init__(self, config):
+        super().__init__()
+        self.prefix_projection = config.prefix_projection
+        token_dim = config.token_dim
+        num_layers = config.num_layers
+        encoder_hidden_size = config.encoder_hidden_size
+        num_virtual_tokens = config.num_virtual_tokens
+        if self.prefix_projection and not config.inference_mode:
+            # Use a two-layer MLP to encode the prefix
+            self.embedding = torch.nn.Embedding(num_virtual_tokens, token_dim)
+            self.transform = torch.nn.Sequential(
+                torch.nn.Linear(token_dim, encoder_hidden_size),
+                torch.nn.Tanh(),
+                torch.nn.Linear(encoder_hidden_size, num_layers * 2 * token_dim),
+            )
+        else:
+            self.embedding = torch.nn.Embedding(num_virtual_tokens, num_layers * 2 * token_dim)
+
+    def forward(self, prefix: torch.Tensor):
+        if self.prefix_projection:
+            prefix_tokens = self.embedding(prefix)
+            past_key_values = self.transform(prefix_tokens)
+        else:
+            past_key_values = self.embedding(prefix)
+        return past_key_values

SVFT-main/LLM-Adapters/peft/src/peft/tuners/prompt_tuning.py

@@ -0,0 +1,120 @@
+# coding=utf-8
+# Copyright 2023-present the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import enum
+import math
+from dataclasses import dataclass, field
+from typing import Optional, Union
+
+import torch
+
+from ..utils import PeftType, PromptLearningConfig
+
+
+class PromptTuningInit(str, enum.Enum):
+    TEXT = "TEXT"
+    RANDOM = "RANDOM"
+
+
+@dataclass
+class PromptTuningConfig(PromptLearningConfig):
+    """
+    This is the configuration class to store the configuration of a [`~peft.PromptEmbedding`].
+
+    Args:
+        prompt_tuning_init (Union[[`PromptTuningInit`], `str`]): The initialization of the prompt embedding.
+        prompt_tuning_init_text ( Optional[`str`]): The text to initialize the prompt embedding.
+            Only used if `prompt_tuning_init` is `TEXT`
+        tokenizer_name_or_path ( Optional[`str`]): The name or path of the tokenizer.
+            Only used if `prompt_tuning_init` is `TEXT`
+    """
+
+    prompt_tuning_init: Union[PromptTuningInit, str] = field(
+        default=PromptTuningInit.RANDOM,
+        metadata={"help": "How to initialize the prompt tuning parameters"},
+    )
+    prompt_tuning_init_text: Optional[str] = field(
+        default=None,
+        metadata={
+            "help": "The text to use for prompt tuning initialization. Only used if prompt_tuning_init is `TEXT`"
+        },
+    )
+    tokenizer_name_or_path: Optional[str] = field(
+        default=None,
+        metadata={
+            "help": "The tokenizer to use for prompt tuning initialization. Only used if prompt_tuning_init is `TEXT`"
+        },
+    )
+
+    def __post_init__(self):
+        self.peft_type = PeftType.PROMPT_TUNING
+
+
+class PromptEmbedding(torch.nn.Module):
+    """
+    The model to encode virtual tokens into prompt embeddings.
+
+    Args:
+        config ([`PromptTuningConfig`]): The configuration of the prompt embedding.
+        word_embeddings (`torch.nn.Module`): The word embeddings of the base transformer model.
+
+    **Attributes**:
+        **embedding** (`torch.nn.Embedding`) -- The embedding layer of the prompt embedding.
+
+    Example::
+
+        >>> from peft import PromptEmbedding, PromptTuningConfig >>> config = PromptTuningConfig(
+                peft_type="PROMPT_TUNING", task_type="SEQ_2_SEQ_LM", num_virtual_tokens=20, token_dim=768,
+                num_transformer_submodules=1, num_attention_heads=12, num_layers=12, prompt_tuning_init="TEXT",
+                prompt_tuning_init_text="Predict if sentiment of this review is positive, negative or neutral",
+                tokenizer_name_or_path="t5-base",
+            )
+        >>> # t5_model.shared is the word embeddings of the base model >>> prompt_embedding = PromptEmbedding(config,
+        t5_model.shared)
+
+
+    Input Shape: (batch_size, total_virtual_tokens)
+
+    Output Shape: (batch_size, total_virtual_tokens, token_dim)
+    """
+
+    def __init__(self, config, word_embeddings):
+        super().__init__()
+
+        total_virtual_tokens = config.num_virtual_tokens * config.num_transformer_submodules
+        self.embedding = torch.nn.Embedding(total_virtual_tokens, config.token_dim)
+        if config.prompt_tuning_init == PromptTuningInit.TEXT:
+            from transformers import AutoTokenizer
+
+            tokenizer = AutoTokenizer.from_pretrained(config.tokenizer_name_or_path)
+            init_text = config.prompt_tuning_init_text
+            init_token_ids = tokenizer(init_text)["input_ids"]
+            # Trim or iterate until num_text_tokens matches total_virtual_tokens
+            num_text_tokens = len(init_token_ids)
+            if num_text_tokens > total_virtual_tokens:
+                init_token_ids = init_token_ids[:total_virtual_tokens]
+            elif num_text_tokens < total_virtual_tokens:
+                num_reps = math.ceil(total_virtual_tokens / num_text_tokens)
+                init_token_ids = init_token_ids * num_reps
+            init_token_ids = init_token_ids[:total_virtual_tokens]
+
+            word_embedding_weights = word_embeddings(torch.LongTensor(init_token_ids)).detach().clone()
+            word_embedding_weights = word_embedding_weights.to(torch.float32)
+            self.embedding.weight = torch.nn.Parameter(word_embedding_weights)
+
+    def forward(self, indices):
+        # Just get embeddings
+        prompt_embeddings = self.embedding(indices)
+        return prompt_embeddings

SVFT-main/LLM-Adapters/peft/src/peft/utils/__init__.py

@@ -0,0 +1,30 @@
+# flake8: noqa
+# There's no way to ignore "F401 '...' imported but unused" warnings in this
+# module, but to preserve other warnings. So, don't check this module at all
+
+# coding=utf-8
+# Copyright 2023-present the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from .adapters_utils import CONFIG_NAME, WEIGHTS_NAME
+from .config import PeftConfig, PeftType, PromptLearningConfig, TaskType
+from .other import (
+    TRANSFORMERS_MODELS_TO_PREFIX_TUNING_POSTPROCESS_MAPPING,
+    _set_trainable,
+    bloom_model_postprocess_past_key_value,
+    prepare_model_for_int8_training,
+    shift_tokens_right,
+    transpose,
+)
+from .save_and_load import get_peft_model_state_dict, set_peft_model_state_dict

SVFT-main/LLM-Adapters/peft/src/peft/utils/adapters_utils.py

@@ -0,0 +1,18 @@
+# coding=utf-8
+# Copyright 2023-present the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+WEIGHTS_NAME = "adapter_model.bin"
+CONFIG_NAME = "adapter_config.json"
+
+# TODO: add automapping and superclass here?

SVFT-main/LLM-Adapters/peft/src/peft/utils/config.py

@@ -0,0 +1,169 @@
+# coding=utf-8
+# Copyright 2023-present the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import enum
+import json
+import os
+from dataclasses import asdict, dataclass, field
+from typing import Optional, Union
+
+from huggingface_hub import hf_hub_download
+from transformers.utils import PushToHubMixin
+
+from .adapters_utils import CONFIG_NAME
+
+
+class PeftType(str, enum.Enum):
+    PROMPT_TUNING = "PROMPT_TUNING"
+    P_TUNING = "P_TUNING"
+    PREFIX_TUNING = "PREFIX_TUNING"
+    LORA = "LORA"
+    BOTTLENECK = "BOTTLENECK"
+
+
+
+class TaskType(str, enum.Enum):
+    SEQ_CLS = "SEQ_CLS"
+    SEQ_2_SEQ_LM = "SEQ_2_SEQ_LM"
+    CAUSAL_LM = "CAUSAL_LM"
+    TOKEN_CLS = "TOKEN_CLS"
+
+
+@dataclass
+class PeftConfigMixin(PushToHubMixin):
+    r"""
+    This is the base configuration class for PEFT adapter models. It contains all the methods that are common to all
+    PEFT adapter models. This class inherits from `transformers.utils.PushToHubMixin` which contains the methods to
+    push your model to the Hub. The method `save_pretrained` will save the configuration of your adapter model in a
+    directory. The method `from_pretrained` will load the configuration of your adapter model from a directory.
+
+    Args:
+        peft_type (Union[[`~peft.utils.config.PeftType`], `str`]): The type of Peft method to use.
+    """
+    peft_type: Optional[PeftType] = field(default=None, metadata={"help": "The type of PEFT model."})
+
+    @property
+    def __dict__(self):
+        return asdict(self)
+
+    def to_dict(self):
+        return self.__dict__
+
+    def save_pretrained(self, save_directory, **kwargs):
+        r"""
+        This method saves the configuration of your adapter model in a directory.
+
+        Args:
+            save_directory (`str`):
+                The directory where the configuration will be saved.
+            **kwargs:
+                Additional keyword arguments passed along to the `transformers.utils.PushToHubMixin.push_to_hub`
+                method.
+        """
+        if os.path.isfile(save_directory):
+            raise AssertionError(f"Provided path ({save_directory}) should be a directory, not a file")
+
+        os.makedirs(save_directory, exist_ok=True)
+
+        output_dict = self.__dict__
+        output_path = os.path.join(save_directory, CONFIG_NAME)
+
+        # save it
+        with open(output_path, "w") as writer:
+            writer.write(json.dumps(output_dict, indent=2, sort_keys=True))
+
+    @classmethod
+    def from_pretrained(cls, pretrained_model_name_or_path, **kwargs):
+        r"""
+        This method loads the configuration of your adapter model from a directory.
+
+        Args:
+            pretrained_model_name_or_path (`str`):
+                The directory or the hub-id where the configuration is saved.
+            **kwargs:
+                Additional keyword arguments passed along to the child class initialization.
+        """
+        if os.path.isfile(os.path.join(pretrained_model_name_or_path, CONFIG_NAME)):
+            config_file = os.path.join(pretrained_model_name_or_path, CONFIG_NAME)
+        else:
+            try:
+                config_file = hf_hub_download(pretrained_model_name_or_path, CONFIG_NAME)
+            except Exception:
+                raise ValueError(f"Can't find config.json at '{pretrained_model_name_or_path}'")
+
+        loaded_attributes = cls.from_json_file(config_file)
+
+        config = cls(**kwargs)
+
+        for key, value in loaded_attributes.items():
+            if hasattr(config, key):
+                setattr(config, key, value)
+
+        return config
+
+    @classmethod
+    def from_json_file(cls, path_json_file, **kwargs):
+        r"""
+        Loads a configuration file from a json file.
+
+        Args:
+            path_json_file (`str`):
+                The path to the json file.
+        """
+        with open(path_json_file, "r") as file:
+            json_object = json.load(file)
+
+        return json_object
+
+
+@dataclass
+class PeftConfig(PeftConfigMixin):
+    """
+    This is the base configuration class to store the configuration of a :class:`~peft.PeftModel`.
+
+    Args:
+        peft_type (Union[[`~peft.utils.config.PeftType`], `str`]): The type of Peft method to use.
+        task_type (Union[[`~peft.utils.config.TaskType`], `str`]): The type of task to perform.
+        inference_mode (`bool`, defaults to `False`): Whether to use the Peft model in inference mode.
+    """
+
+    base_model_name_or_path: str = field(default=None, metadata={"help": "The name of the base model to use."})
+    peft_type: Union[str, PeftType] = field(default=None, metadata={"help": "Peft type"})
+    task_type: Union[str, TaskType] = field(default=None, metadata={"help": "Task type"})
+    inference_mode: bool = field(default=False, metadata={"help": "Whether to use inference mode"})
+
+
+@dataclass
+class PromptLearningConfig(PeftConfig):
+    """
+    This is the base configuration class to store the configuration of a Union[[`~peft.PrefixTuning`],
+    [`~peft.PromptEncoder`], [`~peft.PromptTuning`]].
+
+    Args:
+        num_virtual_tokens (`int`): The number of virtual tokens to use.
+        token_dim (`int`): The hidden embedding dimension of the base transformer model.
+        num_transformer_submodules (`int`): The number of transformer submodules in the base transformer model.
+        num_attention_heads (`int`): The number of attention heads in the base transformer model.
+        num_layers (`int`): The number of layers in the base transformer model.
+    """
+
+    num_virtual_tokens: int = field(default=None, metadata={"help": "Number of virtual tokens"})
+    token_dim: int = field(
+        default=None, metadata={"help": "The hidden embedding dimension of the base transformer model"}
+    )
+    num_transformer_submodules: Optional[int] = field(
+        default=None, metadata={"help": "Number of transformer submodules"}
+    )
+    num_attention_heads: Optional[int] = field(default=None, metadata={"help": "Number of attention heads"})
+    num_layers: Optional[int] = field(default=None, metadata={"help": "Number of transformer layers"})

SVFT-main/LLM-Adapters/peft/src/peft/utils/other.py

@@ -0,0 +1,159 @@
+# coding=utf-8
+# Copyright 2023-present the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import torch
+
+
+# needed for prefix-tuning of bloom model
+def bloom_model_postprocess_past_key_value(past_key_values):
+    past_key_values = torch.cat(past_key_values)
+    total_layers, batch_size, num_attention_heads, num_virtual_tokens, head_dim = past_key_values.shape
+    keys = past_key_values[: total_layers // 2]
+    keys = keys.transpose(2, 3).reshape(
+        total_layers // 2, batch_size * num_attention_heads, head_dim, num_virtual_tokens
+    )
+    values = past_key_values[total_layers // 2 :]
+    values = values.reshape(total_layers // 2, batch_size * num_attention_heads, num_virtual_tokens, head_dim)
+
+    return tuple(zip(keys, values))
+
+
+def prepare_model_for_int8_training(
+    model, output_embedding_layer_name="lm_head", use_gradient_checkpointing=True, layer_norm_names=["layer_norm"]
+):
+    r"""
+    This method wrapps the entire protocol for preparing a model before running a training. This includes:
+        1- Cast the layernorm in fp32 2- making output embedding layer require grads 3- Add the upcasting of the lm
+        head to fp32
+
+    Args:
+        model, (`transformers.PreTrainedModel`):
+            The loaded model from `transformers`
+    """
+    loaded_in_8bit = getattr(model, "is_loaded_in_8bit", False)
+
+    for name, param in model.named_parameters():
+        # freeze base model's layers
+        param.requires_grad = False
+
+        if loaded_in_8bit:
+            # cast layer norm in fp32 for stability for 8bit models
+            if param.ndim == 1 and any(layer_norm_name in name for layer_norm_name in layer_norm_names):
+                param.data = param.data.to(torch.float32)
+
+    if loaded_in_8bit and use_gradient_checkpointing:
+        # For backward compatibility
+        if hasattr(model, "enable_input_require_grads"):
+            model.enable_input_require_grads()
+        else:
+
+            def make_inputs_require_grad(module, input, output):
+                output.requires_grad_(True)
+
+            model.get_input_embeddings().register_forward_hook(make_inputs_require_grad)
+
+        # enable gradient checkpointing for memory efficiency
+        model.gradient_checkpointing_enable()
+
+    if hasattr(model, output_embedding_layer_name):
+        output_embedding_layer = getattr(model, output_embedding_layer_name)
+        input_dtype = output_embedding_layer.weight.dtype
+
+        class CastOutputToFloat(torch.nn.Sequential):
+            r"""
+            Manually cast to the expected dtype of the lm_head as sometimes there is a final layer norm that is casted
+            in fp32
+
+            """
+
+            def forward(self, x):
+                return super().forward(x.to(input_dtype)).to(torch.float32)
+
+        setattr(model, output_embedding_layer_name, CastOutputToFloat(output_embedding_layer))
+
+    return model
+
+
+TRANSFORMERS_MODELS_TO_PREFIX_TUNING_POSTPROCESS_MAPPING = {
+    "bloom": bloom_model_postprocess_past_key_value,
+}
+
+
+# copied from transformers.models.bart.modeling_bart
+def shift_tokens_right(input_ids: torch.Tensor, pad_token_id: int, decoder_start_token_id: int):
+    """
+    Shift input ids one token to the right.
+
+    Args:
+        input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): input ids
+        pad_token_id (`int`): The id of the `padding` token.
+        decoder_start_token_id (`int`): The id of the `start` token.
+    """
+    shifted_input_ids = input_ids.new_zeros(input_ids.shape)
+    shifted_input_ids[:, 1:] = input_ids[:, :-1].clone()
+    shifted_input_ids[:, 0] = decoder_start_token_id
+
+    if pad_token_id is None:
+        raise ValueError("self.model.config.pad_token_id has to be defined.")
+    # replace possible -100 values in labels by `pad_token_id`
+    shifted_input_ids.masked_fill_(shifted_input_ids == -100, pad_token_id)
+
+    return shifted_input_ids
+
+
+def _set_trainable(model):
+    if model.modules_to_save is not None:
+        for name, param in model.named_parameters():
+            if any(module_name in name for module_name in model.modules_to_save):
+                param.requires_grad = True
+
+
+def fsdp_auto_wrap_policy(model):
+    import functools
+    import os
+
+    from accelerate import FullyShardedDataParallelPlugin
+    from torch.distributed.fsdp.wrap import _or_policy, lambda_auto_wrap_policy, transformer_auto_wrap_policy
+
+    from ..tuners import PrefixEncoder, PromptEmbedding, PromptEncoder
+
+    def lambda_policy_fn(module):
+        if (
+            len(list(module.named_children())) == 0
+            and getattr(module, "weight", None) is not None
+            and module.weight.requires_grad
+        ):
+            return True
+        return False
+
+    lambda_policy = functools.partial(lambda_auto_wrap_policy, lambda_fn=lambda_policy_fn)
+    transformer_wrap_policy = functools.partial(
+        transformer_auto_wrap_policy,
+        transformer_layer_cls=(
+            PrefixEncoder,
+            PromptEncoder,
+            PromptEmbedding,
+            FullyShardedDataParallelPlugin.get_module_class_from_name(
+                model, os.environ.get("FSDP_TRANSFORMER_CLS_TO_WRAP", "")
+            ),
+        ),
+    )
+
+    auto_wrap_policy = functools.partial(_or_policy, policies=[lambda_policy, transformer_wrap_policy])
+    return auto_wrap_policy
+
+
+def transpose(weight, fan_in_fan_out):
+    return weight.T if fan_in_fan_out else weight

SVFT-main/LLM-Adapters/peft/src/peft/utils/save_and_load.py

@@ -0,0 +1,96 @@
+# coding=utf-8
+# Copyright 2023-present the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from .config import PeftType
+
+
+def get_peft_model_state_dict(model, state_dict=None):
+    """
+    Get the state dict of the Peft model.
+
+    Args:
+        model ([`PeftModel`]): The Peft model. When using torch.nn.DistributedDataParallel, DeepSpeed or FSDP,
+        the model should be the underlying model/unwrapped model (i.e. model.module).
+        state_dict (`dict`, *optional*, defaults to `None`):
+            The state dict of the model. If not provided, the state dict of the model
+        will be used.
+    """
+    if state_dict is None:
+        state_dict = model.state_dict()
+    if model.peft_config.peft_type == PeftType.LORA:
+        # to_return = lora_state_dict(model, bias=model.peft_config.bias)
+        # adapted from `https://github.com/microsoft/LoRA/blob/main/loralib/utils.py`
+        # to directly with the state dict which is necessary when using DeepSpeed or FSDP
+        bias = model.peft_config.bias
+        if bias == "none":
+            to_return = {k: state_dict[k] for k in state_dict if "lora_" in k}
+        elif bias == "all":
+            to_return = {k: state_dict[k] for k in state_dict if "lora_" in k or "bias" in k}
+        elif bias == "lora_only":
+            to_return = {}
+            for k in state_dict:
+                if "lora_" in k:
+                    to_return[k] = state_dict[k]
+                    bias_name = k.split("lora_")[0] + "bias"
+                    if bias_name in state_dict:
+                        to_return[bias_name] = state_dict[bias_name]
+        else:
+            raise NotImplementedError
+    elif model.peft_config.peft_type == PeftType.BOTTLENECK:
+        # return the state dict of the model with Bottleneck adapters
+        bias = model.peft_config.bias
+        if bias == "none":
+            to_return = {k: state_dict[k] for k in state_dict if "adapter_" in k}
+        elif bias == "all":
+            to_return = {k: state_dict[k] for k in state_dict if "adapter_" in k or "bias" in k}
+        elif bias == "adapter_only":
+            to_return = {}
+            for k in state_dict:
+                if "adapter_" in k:
+                    to_return[k] = state_dict[k]
+                    bias_name = k.split("adapter_")[0] + "bias"
+                    if bias_name in state_dict:
+                        to_return[bias_name] = state_dict[bias_name]
+        else:
+            raise NotImplementedError
+    else:
+        to_return = {}
+        if model.peft_config.inference_mode:
+            prompt_embeddings = model.prompt_encoder.embedding.weight
+        else:
+            prompt_embeddings = model.get_prompt_embedding_to_save()
+        to_return["prompt_embeddings"] = prompt_embeddings
+    if model.modules_to_save is not None:
+        for key, value in state_dict.items():
+            if any(module_name in key for module_name in model.modules_to_save):
+                to_return[key] = value
+    return to_return
+
+
+def set_peft_model_state_dict(model, peft_model_state_dict):
+    """
+    Set the state dict of the Peft model.
+
+    Args:
+        model ([`PeftModel`]): The Peft model.
+        peft_model_state_dict (`dict`): The state dict of the Peft model.
+    """
+
+    model.load_state_dict(peft_model_state_dict, strict=False)
+    if model.peft_config.peft_type != PeftType.LORA and model.peft_config.peft_type != PeftType.BOTTLENECK:
+        model.prompt_encoder.embedding.load_state_dict(
+            {"weight": peft_model_state_dict["prompt_embeddings"]}, strict=True
+        )
+    return model

SVFT-main/LLM-Adapters/peft/tests/test_config.py

@@ -0,0 +1,96 @@
+# coding=utf-8
+# Copyright 2023-present the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import os
+import tempfile
+import unittest
+
+from peft import LoraConfig, PrefixTuningConfig, PromptEncoderConfig, PromptTuningConfig
+
+
+class PeftConfigTestMixin:
+    all_config_classes = (
+        LoraConfig,
+        PromptEncoderConfig,
+        PrefixTuningConfig,
+        PromptTuningConfig,
+    )
+
+
+class PeftConfigTester(unittest.TestCase, PeftConfigTestMixin):
+    def test_methods(self):
+        r"""
+        Test if all configs have the expected methods. Here we test
+        - to_dict
+        - save_pretrained
+        - from_pretrained
+        - from_json_file
+        """
+        # test if all configs have the expected methods
+        for config_class in self.all_config_classes:
+            config = config_class()
+            self.assertTrue(hasattr(config, "to_dict"))
+            self.assertTrue(hasattr(config, "save_pretrained"))
+            self.assertTrue(hasattr(config, "from_pretrained"))
+            self.assertTrue(hasattr(config, "from_json_file"))
+
+    def test_task_type(self):
+        for config_class in self.all_config_classes:
+            # assert this will not fail
+            _ = config_class(task_type="test")
+
+    def test_save_pretrained(self):
+        r"""
+        Test if the config is correctly saved and loaded using
+        - save_pretrained
+        """
+        for config_class in self.all_config_classes:
+            config = config_class()
+            with tempfile.TemporaryDirectory() as tmp_dirname:
+                config.save_pretrained(tmp_dirname)
+
+                config_from_pretrained = config_class.from_pretrained(tmp_dirname)
+                self.assertEqual(config.to_dict(), config_from_pretrained.to_dict())
+
+    def test_from_json_file(self):
+        for config_class in self.all_config_classes:
+            config = config_class()
+            with tempfile.TemporaryDirectory() as tmp_dirname:
+                config.save_pretrained(tmp_dirname)
+
+                config_from_json = config_class.from_json_file(os.path.join(tmp_dirname, "adapter_config.json"))
+                self.assertEqual(config.to_dict(), config_from_json)
+
+    def test_to_dict(self):
+        r"""
+        Test if the config can be correctly converted to a dict using:
+        - to_dict
+        - __dict__
+        """
+        for config_class in self.all_config_classes:
+            config = config_class()
+            self.assertEqual(config.to_dict(), config.__dict__)
+            self.assertTrue(isinstance(config.to_dict(), dict))
+
+    def test_set_attributes(self):
+        # manually set attributes and check if they are correctly written
+        for config_class in self.all_config_classes:
+            config = config_class(peft_type="test")
+
+            # save pretrained
+            with tempfile.TemporaryDirectory() as tmp_dirname:
+                config.save_pretrained(tmp_dirname)
+
+                config_from_pretrained = config_class.from_pretrained(tmp_dirname)
+                self.assertEqual(config.to_dict(), config_from_pretrained.to_dict())

SVFT-main/LLM-Adapters/peft/tests/test_peft_model.py

@@ -0,0 +1,156 @@
+# coding=utf-8
+# Copyright 2023-present the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import os
+import tempfile
+import unittest
+
+import torch
+from parameterized import parameterized
+from transformers import AutoModelForCausalLM
+
+from peft import (
+    PeftModel,
+    get_peft_model,
+    get_peft_model_state_dict,
+    prepare_model_for_int8_training,
+)
+
+from .testing_common import PeftTestConfigManager
+
+
+# This has to be in the order: model_id, lora_kwargs, prefix_tuning_kwargs, prompt_encoder_kwargs, prompt_tuning_kwargs
+PEFT_MODELS_TO_TEST = [
+    ("hf-internal-testing/tiny-random-OPTForCausalLM", {"target_modules": ["q_proj", "v_proj"]}, {}, {}, {}),
+]
+
+
+class PeftTestMixin:
+    torch_device = "cuda" if torch.cuda.is_available() else "cpu"
+
+
+class PeftModelTester(unittest.TestCase, PeftTestMixin):
+    r"""
+    Test if the PeftModel behaves as expected. This includes:
+    - test if the model has the expected methods
+
+    We use parametrized.expand for debugging purposes to test each model individually.
+    """
+
+    @parameterized.expand(PeftTestConfigManager.get_grid_parameters(PEFT_MODELS_TO_TEST))
+    def test_attributes_parametrized(self, test_name, model_id, config_cls, config_kwargs):
+        self._test_model_attr(model_id, config_cls, config_kwargs)
+
+    def _test_model_attr(self, model_id, config_cls, config_kwargs):
+        model = AutoModelForCausalLM.from_pretrained(model_id)
+        config = config_cls(
+            base_model_name_or_path=model_id,
+            **config_kwargs,
+        )
+        model = get_peft_model(model, config)
+
+        self.assertTrue(hasattr(model, "save_pretrained"))
+        self.assertTrue(hasattr(model, "from_pretrained"))
+        self.assertTrue(hasattr(model, "push_to_hub"))
+
+    def _test_prepare_for_training(self, model_id, config_cls, config_kwargs):
+        model = AutoModelForCausalLM.from_pretrained(model_id).to(self.torch_device)
+        config = config_cls(
+            base_model_name_or_path=model_id,
+            **config_kwargs,
+        )
+        model = get_peft_model(model, config)
+
+        dummy_input = torch.LongTensor([[1, 1, 1]]).to(self.torch_device)
+        dummy_output = model.get_input_embeddings()(dummy_input)
+
+        self.assertTrue(not dummy_output.requires_grad)
+
+        # load with `prepare_model_for_int8_training`
+        model = AutoModelForCausalLM.from_pretrained(model_id).to(self.torch_device)
+        model = prepare_model_for_int8_training(model)
+
+        for param in model.parameters():
+            self.assertTrue(not param.requires_grad)
+
+        config = config_cls(
+            base_model_name_or_path=model_id,
+            **config_kwargs,
+        )
+        model = get_peft_model(model, config)
+
+        # For backward compatibility
+        if hasattr(model, "enable_input_require_grads"):
+            model.enable_input_require_grads()
+        else:
+
+            def make_inputs_require_grad(module, input, output):
+                output.requires_grad_(True)
+
+            model.get_input_embeddings().register_forward_hook(make_inputs_require_grad)
+
+        dummy_input = torch.LongTensor([[1, 1, 1]]).to(self.torch_device)
+        dummy_output = model.get_input_embeddings()(dummy_input)
+
+        self.assertTrue(dummy_output.requires_grad)
+
+    @parameterized.expand(PeftTestConfigManager.get_grid_parameters(PEFT_MODELS_TO_TEST))
+    def test_prepare_for_training_parametrized(self, test_name, model_id, config_cls, config_kwargs):
+        self._test_prepare_for_training(model_id, config_cls, config_kwargs)
+
+    def _test_save_pretrained(self, model_id, config_cls, config_kwargs):
+        model = AutoModelForCausalLM.from_pretrained(model_id)
+        config = config_cls(
+            base_model_name_or_path=model_id,
+            **config_kwargs,
+        )
+        model = get_peft_model(model, config)
+        model = model.to(self.torch_device)
+
+        with tempfile.TemporaryDirectory() as tmp_dirname:
+            model.save_pretrained(tmp_dirname)
+
+            model_from_pretrained = AutoModelForCausalLM.from_pretrained(model_id)
+            model_from_pretrained = PeftModel.from_pretrained(model_from_pretrained, tmp_dirname)
+
+            # check if the state dicts are equal
+            state_dict = get_peft_model_state_dict(model)
+            state_dict_from_pretrained = get_peft_model_state_dict(model_from_pretrained)
+
+            # check if same keys
+            self.assertEqual(state_dict.keys(), state_dict_from_pretrained.keys())
+
+            # check if tensors equal
+            for key in state_dict.keys():
+                self.assertTrue(
+                    torch.allclose(
+                        state_dict[key].to(self.torch_device), state_dict_from_pretrained[key].to(self.torch_device)
+                    )
+                )
+
+            # check if `adapter_model.bin` is present
+            self.assertTrue(os.path.exists(os.path.join(tmp_dirname, "adapter_model.bin")))
+
+            # check if `adapter_config.json` is present
+            self.assertTrue(os.path.exists(os.path.join(tmp_dirname, "adapter_config.json")))
+
+            # check if `pytorch_model.bin` is not present
+            self.assertFalse(os.path.exists(os.path.join(tmp_dirname, "pytorch_model.bin")))
+
+            # check if `config.json` is not present
+            self.assertFalse(os.path.exists(os.path.join(tmp_dirname, "config.json")))
+
+    @parameterized.expand(PeftTestConfigManager.get_grid_parameters(PEFT_MODELS_TO_TEST))
+    def test_save_pretrained(self, test_name, model_id, config_cls, config_kwargs):
+        self._test_save_pretrained(model_id, config_cls, config_kwargs)

SVFT-main/LLM-Adapters/peft/tests/testing_common.py

@@ -0,0 +1,103 @@
+# coding=utf-8
+# Copyright 2023-present the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from collections import OrderedDict
+
+from peft import (
+    LoraConfig,
+    PrefixTuningConfig,
+    PromptEncoderConfig,
+    PromptTuningConfig,
+)
+
+
+CONFIG_CLASSES = (
+    LoraConfig,
+    PrefixTuningConfig,
+    PromptEncoderConfig,
+    PromptTuningConfig,
+)
+CONFIG_TESTING_KWARGS = (
+    {
+        "r": 8,
+        "lora_alpha": 32,
+        "target_modules": None,
+        "lora_dropout": 0.05,
+        "bias": "none",
+        "task_type": "CAUSAL_LM",
+    },
+    {
+        "num_virtual_tokens": 10,
+        "task_type": "CAUSAL_LM",
+    },
+    {
+        "num_virtual_tokens": 10,
+        "encoder_hidden_size": 32,
+        "task_type": "CAUSAL_LM",
+    },
+    {
+        "num_virtual_tokens": 10,
+        "task_type": "CAUSAL_LM",
+    },
+)
+
+CLASSES_MAPPING = {
+    "lora": (LoraConfig, CONFIG_TESTING_KWARGS[0]),
+    "prefix_tuning": (PrefixTuningConfig, CONFIG_TESTING_KWARGS[1]),
+    "prompt_encoder": (PromptEncoderConfig, CONFIG_TESTING_KWARGS[2]),
+    "prompt_tuning": (PromptTuningConfig, CONFIG_TESTING_KWARGS[3]),
+}
+
+
+# Adapted from https://github.com/huggingface/transformers/blob/48327c57182fdade7f7797d1eaad2d166de5c55b/src/transformers/activations.py#LL166C7-L166C22
+class ClassInstantier(OrderedDict):
+    def __getitem__(self, key, *args, **kwargs):
+        # check if any of the kwargs is inside the config class kwargs
+        if any([kwarg in self[key][1] for kwarg in kwargs]):
+            new_config_kwargs = self[key][1].copy()
+            new_config_kwargs.update(kwargs)
+            return (self[key][0], new_config_kwargs)
+
+        return super().__getitem__(key, *args, **kwargs)
+
+    def get_grid_parameters(self, model_list):
+        r"""
+        Returns a list of all possible combinations of the parameters in the config classes.
+        """
+        grid_parameters = []
+        for model_tuple in model_list:
+            model_id, lora_kwargs, prefix_tuning_kwargs, prompt_encoder_kwargs, prompt_tuning_kwargs = model_tuple
+            for key, value in self.items():
+                if key == "lora":
+                    # update value[1] if necessary
+                    if lora_kwargs is not None:
+                        value[1].update(lora_kwargs)
+                elif key == "prefix_tuning":
+                    # update value[1] if necessary
+                    if prefix_tuning_kwargs is not None:
+                        value[1].update(prefix_tuning_kwargs)
+                elif key == "prompt_encoder":
+                    # update value[1] if necessary
+                    if prompt_encoder_kwargs is not None:
+                        value[1].update(prompt_encoder_kwargs)
+                else:
+                    # update value[1] if necessary
+                    if prompt_tuning_kwargs is not None:
+                        value[1].update(prompt_tuning_kwargs)
+                grid_parameters.append((f"test_{model_id}_{key}", model_id, value[0], value[1]))
+
+        return grid_parameters
+
+
+PeftTestConfigManager = ClassInstantier(CLASSES_MAPPING)

SVFT-main/LLM-Adapters/peft/tests/testing_utils.py

@@ -0,0 +1,49 @@
+# coding=utf-8
+# Copyright 2023-present the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import unittest
+
+import torch
+
+
+def require_torch_gpu(test_case):
+    """
+    Decorator marking a test that requires a GPU. Will be skipped when no GPU is available.
+    """
+    if not torch.cuda.is_available():
+        return unittest.skip("test requires GPU")(test_case)
+    else:
+        return test_case
+
+
+def require_torch_multi_gpu(test_case):
+    """
+    Decorator marking a test that requires multiple GPUs. Will be skipped when less than 2 GPUs are available.
+    """
+    if not torch.cuda.is_available() or torch.cuda.device_count() < 2:
+        return unittest.skip("test requires multiple GPUs")(test_case)
+    else:
+        return test_case
+
+
+def require_bitsandbytes(test_case):
+    """
+    Decorator marking a test that requires the bitsandbytes library. Will be skipped when the library is not installed.
+    """
+    try:
+        import bitsandbytes  # noqa: F401
+    except ImportError:
+        return unittest.skip("test requires bitsandbytes")(test_case)
+    else:
+        return test_case

SVFT-main/LLM-Adapters/pyproject.toml

@@ -0,0 +1,8 @@
+[tool.black]
+line-length = 79
+
+[tool.isort]
+include_trailing_comma = true
+line_length = 79
+multi_line_output = 3
+profile = "black"

SVFT-main/LLM-Adapters/requirements.txt

@@ -0,0 +1,9 @@
+accelerate
+appdirs
+bitsandbytes
+black
+black[jupyter]
+datasets
+fire
+git+https://github.com/huggingface/transformers.git
+gradio

SVFT-main/LLM-Adapters/run_commonsense.sh

@@ -0,0 +1,33 @@
+
+
+#SVFT_PLAIN
+WORLD_SIZE=1 CUDA_VISIBLE_DEVICES=0 torchrun --nproc_per_node=1 --master_port=3191 finetune.py \
+  --base_model 'google/gemma-2b' \
+  --data_path './ft-training_set/commonsense_15k.json' \
+  --output_dir './Gemma_2B_svft_CR15K/' \
+  --batch_size 64 \
+  --micro_batch_size 4 \
+  --num_epochs 3 \
+  --learning_rate 5e-2 \
+  --cutoff_len 512\
+  --val_set_size 120 \
+  --adapter_name svft \
+  --off_diag 0 \
+  --pattern "banded" \
+  --lora_target_modules "q_proj","v_proj","k_proj","o_proj","up_proj","down_proj","gate_proj"
+
+#SVFT_Random_d=16
+WORLD_SIZE=1 CUDA_VISIBLE_DEVICES=0 torchrun --nproc_per_node=1 --master_port=3191 finetune.py \
+  --base_model 'google/gemma-2b' \
+  --data_path './ft-training_set/commonsense_15k.json' \
+  --output_dir './Gemma_2B_svft_16diag_random_CR15K/' \
+  --batch_size 64 \
+  --micro_batch_size 4 \
+  --num_epochs 3 \
+  --learning_rate 5e-3 \
+  --cutoff_len 512\
+  --val_set_size 120 \
+  --adapter_name svft \
+  --off_diag 16 \
+  --pattern "random" \
+  --lora_target_modules "q_proj","v_proj","k_proj","o_proj","up_proj","down_proj","gate_proj"

SVFT-main/MetaMath/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

SVFT-main/MetaMath/README.MD

@@ -0,0 +1,172 @@
+# MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models
+
+[![Code License](https://img.shields.io/badge/Code%20License-Apache_2.0-green.svg)](CODE_LICENSE)
+[![Model Weight License](https://img.shields.io/badge/Model%20Weights%20License-LLaMA2-yellow)](MetaMath/LICENSE)
+[![Python 3.9+](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/downloads/release/python-390/)
+
+<p align="center">
+🤗 <a href="https://huggingface.co/meta-math" target="_blank">HF Repo</a> • 📃 <a href="https://arxiv.org/abs/2309.12284" target="_blank">[MetaMath]</a><br>
+</p>
+
+<p align="center" width="100%">
+<a ><img src="./imgs/metamath.svg" alt="MetaMath" style="width: 80%; min-width: 300px; display: block; margin: auto;"></a>
+</p>
+
+
+## News
+- 🔥 Our **MetaMath-Llemma-7B** model achieves  **30.0 pass@1** on the MATH Benchmarks, surpassing all the SOTA open-source LLM in 7B-13B scales! All the training scripts and the model are opened.
+- 🔥 Our **MetaMath-Mistral-7B** model achieves  **77.7 pass@1** on the [GSM8k Benchmarks](https://github.com/openai/grade-school-math), surpassing all the SOTA open-source LLM! All the training scripts and the model are opened.
+- 🔥 The full **MetaMathQA** dataset is now released in the huggingface [MetaMathQA](https://huggingface.co/datasets/meta-math/MetaMathQA/tree/main)!
+- 🔥 We released the GSM8K_Backward dataset is also released in the huggingface [GSM8K_Backward](https://huggingface.co/datasets/meta-math/GSM8K_Backward) to evaluate the reversal mathematical reasoning ability!
+- 🔥 Although the data augmentation for **MetaMathQA** is sourced from **ChatGPT 3.5**, Our **MetaMath-70B** model outperforms the closed-source LLMs **ChatGPT 3.5** on the GSM8K!
+- 🔥 Our **MetaMath-7B** model achieves  **66.5 pass@1** on the [GSM8k Benchmarks](https://github.com/openai/grade-school-math), **11.6** points higher than the SOTA open-source LLM!
+- 🔥 Our **MetaMath-7B** model achieves  **19.8 pass@1** on the [MATH Benchmarks](https://github.com/hendrycks/math), **9.1** points higher than the SOTA open-source LLM!
+
+| Model | Checkpoint | Paper  | GSM8k | MATH  | License|
+| ----- |------| ---- |------|-------| ----- |
+| MetaMath-70B-V1.0 | 🤗 <a href="https://huggingface.co/meta-math/MetaMath-70B-V1.0" target="_blank">HF Link</a> |  📃 <a href="https://arxiv.org/abs/2309.12284" target="_blank">[MetaMath]</a>| **82.3**  |  **26.6**	| <a href="https://ai.meta.com/resources/models-and-libraries/llama-downloads/" target="_blank">Llama 2  </a> |
+| MetaMath-13B-V1.0 | 🤗 <a href="https://huggingface.co/meta-math/MetaMath-13B-V1.0" target="_blank">HF Link</a> |  📃 <a href="https://arxiv.org/abs/2309.12284" target="_blank">[MetaMath]</a>| **72.3**  |  **22.4** | <a href="https://ai.meta.com/resources/models-and-libraries/llama-downloads/" target="_blank">Llama 2 </a> |
+| MetaMath-7B-V1.0 | 🤗 <a href="https://huggingface.co/meta-math/MetaMath-7B-V1.0" target="_blank">HF Link</a>  |  📃 <a href="https://arxiv.org/abs/2309.12284" target="_blank">[MetaMath]</a>| 	 **66.5**  |  **19.8** |  <a href="https://ai.meta.com/resources/models-and-libraries/llama-downloads/" target="_blank">Llama 2  </a>|
+| MetaMath-Mistral-7B | 🤗 <a href="https://huggingface.co/meta-math/MetaMath-Mistral-7B" target="_blank">HF Link</a>  |  📃 <a href="https://arxiv.org/abs/2309.12284" target="_blank">[MetaMath]</a>| 	 **77.7**  |  **28.2** |  <a href="http://www.apache.org/licenses/" target="_blank">Apache License 2.0  </a>|
+| MetaMath-Llemma-7B | 🤗 <a href="https://huggingface.co/meta-math/MetaMath-Llemma-7B" target="_blank">HF Link</a>  |  📃 <a href="https://arxiv.org/abs/2309.12284" target="_blank">[MetaMath]</a>| 	 **69.2**  |  **30.0** |  <a href="http://www.apache.org/licenses/" target="_blank">Apache License 2.0  </a>|
+                                                                                                                                                                                                                                                                                                   
+                                                                                                                                                                                                                                                                                                                                                                             
+
+## Comparing MetaMath with the LLM models.
+
+🔥 Comprehensive Results
+
+| Model               | GSM8k Pass@1 | MATH Pass@1 |
+|---------------------|--------------|-------------|
+| MPT-7B              | 6.8          | 3.0         |
+| Falcon-7B           | 6.8          | 2.3         |
+| LLaMA-1-7B          | 11.0         | 2.9         |
+| LLaMA-2-7B          | 14.6         | 2.5         |
+| MPT-30B             | 15.2         | 3.1         |
+| LLaMA-1-13B         | 17.8         | 3.9         |
+| GPT-Neo-2.7B        | 19.5         | --          |
+| Falcon-40B          | 19.6         | 2.5         |
+| Baichuan-chat-13B   | 23.9         | --          |
+| Vicuna-v1.3-13B     | 27.6         | --          |
+| LLaMA-2-13B         | 28.7         | 3.9         |
+| InternLM-7B         | 31.2         | --          |
+| ChatGLM-2-6B        | 32.4         | --          |
+| GPT-J-6B            | 34.9         | --          |
+| LLaMA-1-33B         | 35.6         | 3.9         |
+| LLaMA-2-34B         | 42.2         | 6.24        |
+| RFT-7B              | 50.3         | --          |
+| LLaMA-1-65B         | 50.9         | 10.6        |
+| Qwen-7B             | 51.6         | --          |
+| WizardMath-7B       | 54.9         | 10.7        |
+| LLaMA-2-70B         | 56.8         | 13.5        |
+| WizardMath-13B      | 63.9         | 14.0        |
+| 🔥 MetaMath-7B         | **66.5**     | **19.8**    |
+| 🔥 MetaMath-13B        | **72.3**     | **22.4**    |
+| 🔥 MetaMath-Mistral-7B | **77.7**     | **28.2**    |
+| 🔥 MetaMath-Llemma-7B  | **69.2**     | **30.0**    |
+| WizardMath-70B      | 81.6         | 22.7        |
+| 🔥 MetaMath-70B        | **82.3**     | **26.6**    |
+
+<h2 id="env">Quick Start</h2>
+
+Clone Metamath and install the required packages:
+
+```bash
+git clone https://github.com/meta-math/MetaMath.git
+cd MetaMath
+pip install -r requirements.txt
+```
+
+If you encounter a Ray installation problem, please run:
+
+```bash
+pip install --upgrade ray
+pip install --upgrade pyarrow
+pip install pandas
+```
+
+<h2 id="Inference">Dataset Usage</h2>
+
+Run the following command to load the data:
+
+```python
+from datasets import load_dataset
+dataset = load_dataset("meta-math/MetaMathQA")
+```
+
+
+<h2 id="train">Training</h2>
+
+you need to prepare the  llama-2 base model and our **MetaMathQA** dataset huggingface [MetaMathQA](https://huggingface.co/datasets/meta-math/MetaMathQA/tree/main)
+
+```
+bash run.sh
+```
+or
+
+```
+CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python3 -m torch.distributed.launch --master_addr ${MASTER_ADDR} --master_port ${MASTER_PORT} --nproc_per_node=8 --use_env train_math.py \
+    --model_name_or_path "meta-llama/Llama-2-7b-hf" \
+    --data_path "path/to/metamathqa" \
+    --data_length 10000000 \
+    --bf16 True \
+    --output_dir "path/to/save" \
+    --num_train_epochs 3 \
+    --per_device_train_batch_size 4 \
+    --per_device_eval_batch_size 4 \
+    --gradient_accumulation_steps 4 \
+    --evaluation_strategy "no" \
+    --save_strategy "steps" \
+    --save_steps 1000 \
+    --save_total_limit 2 \
+    --learning_rate 2e-5 \
+    --weight_decay 0. \
+    --warmup_ratio 0.03 \
+    --lr_scheduler_type "cosine" \
+    --logging_steps 1 \
+    --fsdp "full_shard auto_wrap" \
+    --fsdp_transformer_layer_cls_to_wrap 'LlamaDecoderLayer' \
+    --tf32 True
+```
+
+### Supervised fine-tuning
+
+We supervised fine-tune MetaMath-7B with the following hyperparameters:
+
+| Hyperparameter | LLaMA 2 7B |
+|----------------|-------------|
+| Batch size     | 128         |
+| Learning rate  | 2e-5        |
+| Epochs         | 3           |
+| Max length     | 512         |
+| LR scheduler   | cosine      |
+
+<h2 id="evaluation">Evaluation</h2>
+
+we use the vllm to help the fast generation:
+
+```
+python eval_gsm8k.py --model "path/to/save" --data_file ./data/test/GSM8K_test.jsonl
+python eval_math.py --model "path/to/save" --data_file ./data/test/MATH_test.jsonl
+```
+where the "path/to/save" should be replaced by the finetuned model, you can also download our series of MetaMath models in huggingface:  
+🤗 <a href="https://huggingface.co/meta-math/MetaMath-7B-V1.0" target="_blank">MetaMath 7B</a> 🤗 <a href="https://huggingface.co/meta-math/MetaMath-13B-V1.0" target="_blank">MetaMath 13B</a> 🤗 <a href="https://huggingface.co/meta-math/MetaMath-70B-V1.0" target="_blank">MetaMath 70B</a>
+
+The inference prompt for our MetaMath is:
+```
+"Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\n{instruction}\n\n### Response: Let's think step by step."
+```
+
+Thanks for the open source code of [WizardMath](https://github.com/nlpxucan/WizardLM/tree/main/WizardMath) and [RFT](https://github.com/OFA-Sys/gsm8k-ScRel/tree/main). Some of our codes are based on them.
+
+<h2 id="citation">Citation</h2>
+Please cite the paper if you refer to our model, code, data or paper from MetaMath.
+
+```
+@article{yu2023metamath,
+  title={MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models},
+  author={Yu, Longhui and Jiang, Weisen and Shi, Han and Yu, Jincheng and Liu, Zhengying and Zhang, Yu and Kwok, James T and Li, Zhenguo and Weller, Adrian and Liu, Weiyang},
+  journal={arXiv preprint arXiv:2309.12284},
+  year={2023}
+}
+```

SVFT-main/MetaMath/data/README.md

@@ -0,0 +1,7 @@
+# MetaMathQA Data
+
+## Train Data
+The full **MetaMathQA** dataset is now released in the huggingface [MetaMathQA](https://huggingface.co/datasets/meta-math/MetaMathQA/tree/main)!
+
+## Test Data
+We released the GSM8K_Backward dataset is also released in the huggingface [GSM8K_Backward](https://huggingface.co/datasets/meta-math/GSM8K_Backward) to evaluate the reversal mathematical reasoning ability!

SVFT-main/MetaMath/data/train/README.md

@@ -0,0 +1,3 @@
+ # MetaMathQA
+
+The full **MetaMathQA** dataset is now released in the huggingface [MetaMathQA](https://huggingface.co/datasets/meta-math/MetaMathQA/tree/main)

SVFT-main/MetaMath/eval_gsm8k.py

@@ -0,0 +1,134 @@
+import argparse
+import json
+import re
+import jsonlines
+from fraction import Fraction
+from vllm import LLM, SamplingParams
+import sys
+MAX_INT = sys.maxsize
+
+def is_number(s):
+    try:
+        float(s)
+        return True
+    except ValueError:
+        pass
+    try:
+        import unicodedata
+        unicodedata.numeric(s)
+        return True
+    except (TypeError, ValueError):
+        pass
+    return False
+
+def extract_answer_number(completion):
+    text = completion.split('The answer is: ')
+    if len(text) > 1:
+        extract_ans = text[-1].strip()
+        match = re.search(r'[\-+]?\d*[\.,/]?\d+', extract_ans)
+        if match:
+            if '/' in match.group():
+                denominator = match.group().split('/')[1]
+                numerator = match.group().split('/')[0]
+                if is_number(denominator) == True and is_number(numerator) == True:
+                    if denominator == '0':
+                        return round(float(numerator.replace(',', '')))
+                    else:
+                        frac = Fraction(match.group().replace(',', ''))
+                        num_numerator = frac.numerator
+                        num_denominator = frac.denominator
+                        return round(float(num_numerator / num_denominator))
+                else:
+                    return None
+            else:
+                if float(match.group().replace(',', '')) == float('inf'):
+                    return None
+                return round(float(match.group().replace(',', '')))
+        else:
+            return None
+    else:
+        return None
+
+def batch_data(data_list, batch_size=1):
+    n = len(data_list) // batch_size
+    batch_data = []
+    for i in range(n-1):
+        start = i * batch_size
+        end = (i+1)*batch_size
+        batch_data.append(data_list[start:end])
+
+    last_start = (n-1) * batch_size
+    last_end = MAX_INT
+    batch_data.append(data_list[last_start:last_end])
+    return batch_data
+
+
+def gsm8k_test(model, data_path, start=0, end=MAX_INT, batch_size=1, tensor_parallel_size=1):
+    INVALID_ANS = "[invalid]"
+    gsm8k_ins = []
+    gsm8k_answers = []
+    problem_prompt = (
+        "Below is an instruction that describes a task. "
+        "Write a response that appropriately completes the request.\n\n"
+        "### Instruction:\n{instruction}\n\n### Response: Let's think step by step."
+    )
+    print('promt =====', problem_prompt)
+    with open(data_path,"r+", encoding="utf8") as f:
+        for idx, item in enumerate(jsonlines.Reader(f)):
+            temp_instr = problem_prompt.format(instruction=item["query"])
+            gsm8k_ins.append(temp_instr)
+            temp_ans = item['response'].split('#### ')[1]
+            temp_ans = int(temp_ans.replace(',', ''))
+            gsm8k_answers.append(temp_ans)
+
+    gsm8k_ins = gsm8k_ins[start:end]
+    gsm8k_answers = gsm8k_answers[start:end]
+    print('lenght ====', len(gsm8k_ins))
+    batch_gsm8k_ins = batch_data(gsm8k_ins, batch_size=batch_size)
+
+    stop_tokens = ["Question:", "Question", "USER:", "USER", "ASSISTANT:", "ASSISTANT", "Instruction:", "Instruction", "Response:", "Response"]
+    sampling_params = SamplingParams(temperature=0.0, top_p=1, max_tokens=512, stop=stop_tokens)
+    print('sampleing =====', sampling_params)
+    llm = LLM(model=model,tensor_parallel_size=tensor_parallel_size)
+    result = []
+    res_completions = []
+    for idx, (prompt, prompt_answer) in enumerate(zip(batch_gsm8k_ins, gsm8k_answers)):
+        if isinstance(prompt, list):
+            pass
+        else:
+            prompt = [prompt]
+
+        completions = llm.generate(prompt, sampling_params)
+        for output in completions:
+            prompt = output.prompt
+            generated_text = output.outputs[0].text
+            res_completions.append(generated_text)
+
+    invalid_outputs = []
+    for idx, (prompt, completion, prompt_answer) in enumerate(zip(gsm8k_ins, res_completions, gsm8k_answers)):
+        doc = {'question': prompt}
+        y_pred = extract_answer_number(completion)
+        if y_pred != None:
+            result.append(float(y_pred) == float(prompt_answer))
+        else:
+            result.append(False)
+            temp = {'question': prompt, 'output': completion, 'answer': prompt_answer}
+            invalid_outputs.append(temp)
+    acc = sum(result) / len(result)
+    print('len invalid outputs ====', len(invalid_outputs), ', valid_outputs===', invalid_outputs)
+    print('start===', start, ', end====', end)
+    print('gsm8k length====', len(result), ', gsm8k acc====', acc)
+
+
+def parse_args():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--model", type=str)  # model path
+    parser.add_argument("--data_file", type=str, default='')  # data path
+    parser.add_argument("--start", type=int, default=0) #start index
+    parser.add_argument("--end", type=int, default=MAX_INT)  # end index
+    parser.add_argument("--batch_size", type=int, default=400)  # batch_size
+    parser.add_argument("--tensor_parallel_size", type=int, default=8)  # tensor_parallel_size
+    return parser.parse_args()
+if __name__ == "__main__":
+    args = parse_args()
+    gsm8k_test(model=args.model, data_path=args.data_file, start=args.start, end=args.end, batch_size=args.batch_size, tensor_parallel_size=args.tensor_parallel_size)

SVFT-main/MetaMath/eval_math.py

@@ -0,0 +1,115 @@
+import argparse
+import json
+import pdb
+import jsonlines
+
+import util
+from vllm import LLM, SamplingParams
+import sys
+MAX_INT = sys.maxsize
+INVALID_ANS = "[invalid]"
+
+invalid_outputs = []
+def remove_boxed(s):
+    left = "\\boxed{"
+    try:
+        assert s[:len(left)] == left
+        assert s[-1] == "}"
+        return s[len(left):-1]
+    except:
+        return None
+
+def process_results(doc, completion, answer):
+    split_ans = completion.split('The answer is: ')
+    if len(split_ans) > 1:
+        ans = split_ans[-1]
+        extract_ans_temp = ans.split('.\n')[0]
+        extract_ans_temp = extract_ans_temp.strip()
+        if len(extract_ans_temp)>0 and extract_ans_temp[-1] == '.':
+            extract_ans = extract_ans_temp[0:-1]
+        else:
+            extract_ans = extract_ans_temp
+        extract_ans = extract_ans.strip()
+        if util.is_equiv(extract_ans, answer):
+            return True
+        else:
+            return False
+    else:
+        temp = {'question': doc, 'output': completion, 'answer': answer}
+        invalid_outputs.append(temp)
+        return False
+def batch_data(data_list, batch_size=1):
+    n = len(data_list) // batch_size
+    batch_data = []
+    for i in range(n-1):
+        start = i * batch_size
+        end = (i+1)*batch_size
+        batch_data.append(data_list[start:end])
+
+    last_start = (n-1) * batch_size
+    last_end = MAX_INT
+    batch_data.append(data_list[last_start:last_end])
+    return batch_data
+
+def test_hendrycks_math(model, data_path, start=0, end=MAX_INT, batch_size=1, tensor_parallel_size=1):
+    hendrycks_math_ins = []
+    hendrycks_math_answers = []
+    problem_prompt = (
+        "Below is an instruction that describes a task. "
+        "Write a response that appropriately completes the request.\n\n"
+        "### Instruction:\n{instruction}\n\n### Response: Let's think step by step."
+    )
+    print('promt =====', problem_prompt)
+    with open(data_path, "r+", encoding="utf8") as f:
+        for idx, item in enumerate(jsonlines.Reader(f)):
+            temp_instr = problem_prompt.format(instruction=item["instruction"])
+            hendrycks_math_ins.append(temp_instr)
+            solution = item['output']
+            temp_ans = remove_boxed(util.last_boxed_only_string(solution))
+            hendrycks_math_answers.append(temp_ans)
+
+    print('total length ===', len(hendrycks_math_ins))
+    hendrycks_math_ins = hendrycks_math_ins[start:end]
+    hendrycks_math_answers = hendrycks_math_answers[start:end]
+    print('lenght ====', len(hendrycks_math_ins))
+    batch_hendrycks_math_ins = batch_data(hendrycks_math_ins, batch_size=batch_size)
+
+    stop_tokens = ["Question:", "Question", "USER:", "USER", "ASSISTANT:", "ASSISTANT", "Instruction:", "Instruction", "Response:", "Response"]
+    sampling_params = SamplingParams(temperature=0, top_p=1, max_tokens=2048, stop=stop_tokens)
+    print('sampleing =====', sampling_params)
+    llm = LLM(model=model,tensor_parallel_size=tensor_parallel_size)
+    res_completions = []
+    for idx, (prompt, prompt_answer) in enumerate(zip(batch_hendrycks_math_ins, hendrycks_math_answers)):
+        if isinstance(prompt, list):
+            pass
+        else:
+            prompt = [prompt]
+        completions = llm.generate(prompt, sampling_params)
+        for output in completions:
+            prompt_temp = output.prompt
+            generated_text = output.outputs[0].text
+            res_completions.append(generated_text)
+
+    results = []
+    for idx, (prompt, completion, prompt_answer) in enumerate(zip(hendrycks_math_ins, res_completions, hendrycks_math_answers)):
+        res = process_results(prompt, completion, prompt_answer)
+        results.append(res)
+
+    acc = sum(results) / len(results)
+    print('len invalid outputs ====', len(invalid_outputs), ', valid_outputs===', invalid_outputs)
+    print('start===', start, ', end====',end)
+    print('length====', len(results), ', acc====', acc)
+
+def parse_args():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--model", type=str, default='')  # model path
+    parser.add_argument("--data_file", type=str, default='')  # data path
+    parser.add_argument("--start", type=int, default=0) #start index
+    parser.add_argument("--end", type=int, default=MAX_INT)  # end index
+    parser.add_argument("--batch_size", type=int, default=400)  # batch_size
+    parser.add_argument("--tensor_parallel_size", type=int, default=8)  # tensor_parallel_size
+    return parser.parse_args()
+
+if __name__ == "__main__":
+    args = parse_args()
+    test_hendrycks_math(model=args.model, data_path=args.data_file, start=args.start, end=args.end, batch_size=args.batch_size, tensor_parallel_size=args.tensor_parallel_size)

SVFT-main/MetaMath/requirements.txt

@@ -0,0 +1,16 @@
+transformers>=4.34.0
+wandb==0.15.3
+torch==2.0.1
+sentencepiece==0.1.99
+tokenizers==0.13.3
+accelerate==0.21.0
+bitsandbytes==0.40.0
+vllm
+fraction
+tqdm
+numpy
+fire
+openai
+scipy
+jsonlines
+pandas